Technical, Cost Effectiveness and Sustainability Audit, March 2016

National Community Driven Development Project

(NCDDP)

Final Report

Findings and Recommendations

Neil Neate, P.Eng.

Consultant to Department of Rural Development, Government of Myanmar

March 29, 2016 

Technical, Cost Effectiveness and Sustainability Audit, NCDDP, March 2016

Table of Contents

Executive Summary………………………………………………………………………………..… 3

1 Background……………………………………………………………………………………………… 11

2 Technical, Cost Effectiveness and Sustainability Audit Scope………………………. 12

3 Technical Evaluation Team Members …………………………..…………………………… 16

4 Site Selection Procedure and Sampling Methodology ………………………………… 17

5 Technical Evaluation Methodologies …………………………………………………………. 20

5.1 NCDDP Sub-Project Types………………….……………………………………….……… 20

5.2 Technical Evaluation Field Instruments………………………………………………. 20

5.3 Technical Rating System ……………………………………………………………………. 22

5.4 Quality Ratings and Other Criteria ……………………………………………………… 23

5.5 Field Checklist Data Input…………………………………………………………………… 24

6 NCDDP Sub-Projects Evaluated……………………………………….…………………………. 26

7 Technical Design Quality – Findings …………………………………………..……………… 28

8 Cost Effectiveness…………………………………………………..……………..………..…………. 46

9 Compliance with Environmental and Social Safeguards……………..…….…………. 62

10 Operation and Maintenance/Sustainability……………..……………..……………..……. 66

11 Best Practices and Recommendations………………………………………………………… 79

11.1 Building Sub-Project Type………………………………………………………………… 79

11.2 Bridge……………………………………………………………………………………………… 82

11.3 Water Supply…………………………………………………………………………………… 83

11.4 Road, Drainage and Retaining Wall…………………………………………………… 85

11.5 Electricity………………………………………………………………………………………… 87

12 Summary of Findings and Conclusions……………………………………………………….. 89

ANNEX 1 – RECOMMENDATIONS OF THE TECHNICAL AUDIT

ANNEX 2 –SAMPLE TECHNICAL EVALUATION FIELD INSTRUMENT

ANNEX 3 – SUB-PROJECT COMPONENT/ASPECT LIST

ANNEX 4 – SUB-PROJECTS EVALUATED

ANNEX 5 – COMMENTARY FROM ALL SUB-PROJECTS

ANNEX 6 – BRIEF SUB-PROJECT DESCRIPTIONS AND PHOTOGRAPHS

 

Executive Summary

The National Community Driven Development Project (NCDDP) was initially established in 2013 by the Government of Myanmar with support from the World Bank. In 2015, the project was scaled up, with additional financing support from the Government of Myanmar, the World Bank as well as the Government of Italy. The NCDDP is implemented by the Department of Rural Development under the Ministry of Agriculture, Livestock, and Irrigation. The project provides grants to village tracts to finance the construction of rural, community-level civil infrastructure. At the time of the conduct of this technical audit, the project had completed two cycles and financed over 2,000 sub-projects in 9 townships across the country .

This technical evaluation and audit was undertaken to independently assess the technical quality, project facilitation, cost effectiveness, compliance with environmental and social safeguards, and the operation and maintenance/sustainability of a random representative sample of infrastructures that have been completed. A summary of best practices and ‘lessons learned’ was also sought from these inspections and interviews with village stakeholders. The random sampling was based on sub-projects from Year 1 and Year 2; from all nine Townships served by NCDDP; from a mix of remote and not remote villages; and from a representative number of each sub-project type. A total of 210 NCDDP sub-projects or roughly 10% of sub-projects were evaluated during this exercise.

The cost effectiveness of NCDDP investments was determined by including visits and evaluations of comparable pieces of infrastructures financed by other entities.

The technical evaluation was conducted by an independent technical consultant, Neil Neate, P.Eng. Neil was assisted by two Myanmar consulting civil engineers and eight civil/mechanical engineers who are employed by DRD but not working on the NCDDP.

The selection of 210 NCDDP sub-projects was performed using a stratified random sampling method, employing the following criteria:

 There was proportional representation of SPs within each Township;

 There was proportional representation of SPs by infrastructure type;

 Greater than 50% of the selected sites were considered remote; and

 Thirty of the SPs evaluated were constructed in Year 1, enabling the operation and maintenance of infrastructures to be evaluated.

Five types of sub-projects were evaluated: Building, represented by schools, health centers and village halls; Bridge; Water Supply; Road; Electricity. Each SP type was evaluated using a set of Field Tools that were similar in scope and style but differed from one another in the type of information gathered. The Building Technical Rating Field Tool, for example, collected data in regards to concrete practices, wall, column and roof information, etc.; while the Water Supply Tool examined piping, reservoirs and public tapstands. There were five Field Tools for each SP type: Field Tool 1 – Technical Quality Evaluation; 2 – Cost Effectiveness; 3 – Environmental and Social Safeguards; 4 – Operation/Maintenance and Sustainability; and 5 – Key Issues.

Table ES.1 – Sampling Framework

SP Main Type Number of SP Evaluated – Year 1 Number of SP Evaluated – Year 2 Total Number of SP Evaluated

Building 9 55 64

Bridge 2 13 15

Water Supply 4 32 36

Road 11 61 72

Electricity 4 19 23

Total 30 180 210

Technical Design Quality

The entire sub-project ‘packages’ (preparation, design, implementation and follow-up) were evaluated using the WB six-point quality rating system (Highly Satisfactory, Moderately Satisfactory, Satisfactory, Moderately Unsatisfactory, Unsatisfactory, and Highly Unsatisfactory). It was found that 4% of the SPs are Highly Satisfactory with a further 88% rated Satisfactory. The evaluation teams found 7% to be Moderately Satisfactory, and only 1% (2 SP) were considered Highly Unsatisfactory.

Considering the aggregated total of all sub-projects evaluated, it was found that 78% of the technical components of the structures have been constructed in accordance with the plans and specifications as set out in the sub-project proposals and considered to Meet Specification with a further 19% rated Slightly Below Spec in terms of meeting the intent of the sub-project proposal. Only 3% of technical ratings were Below Specification.

There were differences found in the technical quality amongst sub-project type. Bridge sub-projects’ components were found to Meet Spec 91% of the time. Water supply and building SPs were both rated as 79% Meet Spec, with a further 18% and 19%, respectively, rated Slightly Below Spec for these two sub-project types. Road SPs were rated at 75% Meet Spec, while more problems were observed at electrical schemes: 57% Meet Spec and 42% Slightly Below Spec.

Since these technical ratings have been assigned to specific components or aspects of each of these infrastructures, the identification of problem areas is possible and recommendations are provided to improve the technical quality of these items.

The data was also sorted to determine if there are any apparent trends in technical quality based upon when the SP was constructed.

Table ES.3: Construction Year, Aggregate of Ratings for All SPs, All Components

Meets Spec Slightly Below Below Spec

Year 1 (30 SPs) 70% 29% 1%

Year 2 (180) 79% 18% 3%

The table shows that the overall technical quality of SPs has increased from 70% to 79% of components meeting specification. This may be an indication of an increasingly knowledgeable staff that is gaining experience.

Technical Facilitation

The frequency of technical facilitator visits to SP sites was noted down during the technical evaluation visits and was found to be an average of five construction facilitation visits for each SP. The technical quality of SPs was also compared with the degree of remoteness of each village and found to be roughly equal across the nation.

Table ES.2 – Remoteness, Aggregate of Ratings for All SPs, All Components

Meet Spec Slightly Below Below Spec

Not Remote (55 SP) 78% 21% 1%

Remote (87 SP) 78% 18% 5%

Very Remote (59 SP) 79% 19% 2%

Extremely Remote (9 SP) 75% 25% 0%

It can be seen that very little fluctuation of the aggregated sum of ratings is evident when comparing SPs’ degree of remoteness. This indicates that NCDDP technical facilitation efforts produce roughly equal results regardless of the SP villages’ remoteness.

Cost Effectiveness

NCDDP’s buildings, concrete road works and solar panel electrification schemes were found to be equally cost effective as compared to similar constructions by other agencies. NCDDP unit costs for these types of infrastructure are in line with those of other projects.

NCDDP roads constructed of earth and gravel were found to be more cost effective than similar constructions by other entities.

The audit’s sampling of bridges, both NCDDP and comparable works by others, was small and featured a variety of construction methodologies and materials. Nevertheless, an analysis of the information indicates that NCDDP bridge construction costs are in line with those of DRD and therefore are cost effective.

No comparable infrastructure by others was found to contrast with NCDDP’s gravity-fed water supply program nor macadam road works, so conclusions cannot be drawn for these aspects of the Project’s works. NCDDP’s borehole program, however, was found to be cost effective when compared with a similar groundwater well scheme by DRD.

A comparison of the Community Force Account (CFA) versus Contractor construction implementation modalities indicates that CFA will be more cost competitive for bridge and solar panel SPs but less so for roads and electrical grid extensions. The two modalities are roughly equal when considering buildings and mini-hydro SPs, and no firm conclusions can be drawn for water systems.

As would be expected, community contributions, in many cases labour and locally sourced materials, were found to enhance the cost effectiveness of all NCDDP sub-projects, particularly concrete road schemes.

Compliance with Environmental and Social Safeguards

The data collected at the NCDDP SP villages indicates that the Environmental Codes of Practice, the Operation Manual standards for verification and monitoring, and the social screening checklists are being used and followed in a majority of SP village files – 96% of sub-project files contained appropriately completed ECoP documentation, with a greater majority making use of the social screening checklist (99%).

Technical inspections of the sub-project sites during the evaluation showed that 96% of environmental considerations had been appropriately handled on the ground, during and after construction.

Voluntary land donations have taken place for 17% of the SPs evaluated. Proper documentation was found in village files in all cases.

Operation and Maintenance / Sustainability

O&M Committees have been formed and are functioning in 76% of the Year 1 sub-projects evaluated (with electrical sub-projects being the extreme outlier at only 50% of SP sites). The scale of activities undertaken by these committees, both routine maintenance items and major repairs, differs between sub-project types. Roads and water supply sub-projects reported greater numbers of deferred major repairs, perhaps due to expense and capacity problems.

The prevalence of certain routine maintenance activities was measured for each sub-project type. The results of this survey and analysis will inform future O&M training sessions. A great majority of O&M Committees indicate that ongoing capacity development activities have been undertaken. Village sub-project O&M Committees will benefit from an increased awareness of proper O&M techniques.

Only 18% of village committees have instituted a user fee while 23% of villages hold O&M funds in a bank account. Village committees report that all ongoing O&M activities are wholly supported by the villagers, with no inputs from line Ministries or government agencies.

Sub-projects that were undertaken using a CFA implementation modality were found to be more inclined to institute user fees for community infrastructure and are more likely to hold funds in village accounts for future use in operation and maintenance activities.

Major Recommendations

This report provides analysis and a summary of the major problems and challenges associated with the NCDDP construction program. Recommendations of corrective measures and proper construction methodologies are presented throughout the report for specific items. Following are the major recommendations from this study:

• NCDDP should convene a technical sharing session where Township and Union engineering representatives meet to exchange ideas on how SP designs and file documentation can be improved, presenting examples.

• NCDDP engineers and technical staff must check that properly executed as-built drawings are created for all completed SPs. Design changes should be reviewed with senior personnel so that standard template drawings can be altered if warranted.

• A Disaster Risk Management training course should be held to emphasize the responsibility of designers to more fully consider the forces of nature when planning rural infrastructures.

• All NCDDP environmental and social safeguard checklists and forms must be completed for each SP site. Environmental monitoring activities should be ongoing during the SP construction, with notes to file as appropriate. Refresher training courses should emphasize the importance of this documentation.

• NCDDP Engineering Department should provide refresher training sessions to O&M Committees on the 1-year anniversary of the completion of a SP, performing a rigorous inspection of the works and then offering pointers as to how regular periodic maintenance can increase the usefulness and functionality of the infrastructure. Engineering inspections of the systems should take place prior to these sessions so that the course material can be adjusted to suit each individual site.

• The NCDDP should consider revising O&M Committee documentation to stipulate activities that must be undertaken according to a routine schedule, with realistic funds allocated for labour and materials.

• The NCDDP should consider revising O&M Committee documentation to insert specific capital repair estimates. Estimates should be provided appropriate to SP type, for example, roof replacement for buildings, with options described to committees for the funding of such major repair capital works.

• NCDDP should develop a comprehensive set of standard template drawings, designs, details and specifications for all SP types; and train its technical personnel in the use and modification of these standard drawings.

The findings and recommendations from this report will provide direction for the continued success of NCDDP. Following is a table listing the main findings for each section of this technical evaluation.

Topic Main Findings Remarks

1 Technical Design Quality

• Entire Sub-Project ‘Package’ 4% of the SPs are Highly Satisfactory with a further 88% rated Satisfactory.

 

• Individual SP components Total of all sub-project components evaluated, 78% of the technical components of the structures were rated Meets Specification with a further 19% rated Slightly Below Specification.

• Technical Quality from Year 1 to Year 2 Overall technical quality of SPs has increased from 70% to 79% of components Meeting Specification.

• Technical Facilitation Average of five construction facilitation visits for each SP.

2 Cost Effectiveness

• Comparisons with infrastructure by other agencies NCDDP’s buildings, concrete road works, bridge, borehole well and solar panel electrification schemes were found to be equally cost effective.

Roads constructed of earth and gravel were found to be more cost effective. No comparable infrastructure by other agencies was found to contrast with NCDDP’s gravity-fed water supply program nor macadam road works.

• Comparison of Community Force Account (CFA) vs. Contractor implementation methods CFA are more cost competitive for bridge and solar panel SPs but less so for roads and electrical grid extensions. The two modalities are roughly equal for building and mini-hydro SPs. Not enough data was gathered to make any judgment on water supply systems.

• Community contributions Voluntary community contributions, in many cases labour and locally sourced materials, were found to enhance the cost effectiveness of all NCDDP sub-projects.

3 Environmental and Social Safeguards

• Environment Code of Practice 96% of sub-project files contained appropriately completed ECoP documentation, with a greater majority making use of the social screening checklist (99%).

• Environmental inspection 96% of environmental considerations had been appropriately handled on the ground, during and after construction.

• Land donations Voluntary land donations have taken place for 17% of the SPs evaluated. Proper documentation was found in village files in all cases.

 

4 Operation and Maintenance/Sustainability

• O&M Committees O&M Committees have been formed and are functioning in 76% of the Year 1 sub-projects evaluated Only 50% of electrical SPs have a functioning O&M Committee.

• Major repairs Roads and water supply sub-project O&M committees reported greater numbers of deferred major repairs.

• User Fees for O&M use 18% of village committees have instituted a user fee while 23% of villages hold O&M funds in a bank account.

• Comparison of CFA vs. Contractor implementation methods CFA SPs were found more inclined to institute user fees for community infrastructure and more likely to hold funds in village accounts.

 

Technical, Cost Effectiveness and Sustainability Audit, NCDDP, January 2016

Final Report – Findings and Recommendations

1 Background

The National Community Driven Development Project (NCDDP) was established in 2013 by the Government of Myanmar with support from the World Bank. In 2015, the project was scaled up, with additional financing support from the Government of Myanmar, the World Bank and the Government of Italy. The NCDDP is implemented by the Department of Rural Development under the Ministry of Agriculture, Livestock, and Irrigation. The project provides grants to village tracts to finance the construction of rural, community-level civil infrastructure. The project provides facilitation and capacity building to help villagers make choices in an inclusive manner and carry out construction using a transparent process.

The project has completed two cycles and financed over 2,000 sub-projects in 9 townships across the country. The sub-projects identified, selected and implemented by the communities include schools, health clinics, footpaths, jetties, bridges, electrification schemes and other critical community infrastructure. The project has reached about one million people to date and will expand over the next two years to include at least 63 townships, home to approximately seven million people.

The objective of this technical evaluation and audit is to independently assess the technical quality, the cost effectiveness and the sustainability of the rural infrastructure financed by the NCDDP Project. Evaluation teams will also take note of best practices observed and lessons learned at SP sites, in order to make recommendations for future project implementation improvements.

The cost effectiveness of NCDDP investments was determined by including visits and evaluations of comparable pieces of infrastructures financed by other entities.

 

2 Technical, Cost Effectiveness and Sustainability Audit Scope

The main scope of the study is as follows:

2.1 Technical Design Quality of NCDDP SP

The evaluation required an inspection of the infrastructure, examination of SP village files, discussions with village implementation committee members, and comparison of the as-built structure with the approved-for-construction drawings.

The scope of questions to be answered are as follows:

• What is the technical quality of the design? Assess the as built condition in as far as possible as good, fair or poor based on list of key criteria developed for each major type of subproject to be checked as basis for the technical quality assessment.

• What is the quality of materials/inputs and are these consistent with the BOQ in the bidding documents?

• Did the sub-projects follow the technical specifications as designed? Were any critical design elements, such as latrines, dropped?

• What documentation exists to show that the sub-project meets the design and specification requirements?

• Have all technical requirements been met and defects addressed before sub-projects are handed over to communities?

• Did the sub-projects take into account DRM measures? If so, how?

2.2 Cost Effectiveness

This study of cost effectiveness included the evaluation of similar rural infrastructures funded and constructed by other donors or the Government of Myanmar (GoM). These SP are termed “comparable”.

The scope of questions to be answered are as follows:

• How does the budget and unit costs compare between the NCDDP sub-projects and comparable infrastructure built by other parties? Breakdown the NCDDP sub-projects into comparable groups of similar technical quality and utility.

• Are investments implemented through community force account (CFA) more competitive than those implemented by contractors, when the cost of investments, capacity development and supervision, and the cost and quality of O&M, are taken into account?

• Are there community contributions, and if yes, how much, how was it calculated, what forms did these contributions take and what percent of total costs?

• Where community contributions are identified in the sub-project documents,

o Did the contributions actually occur and were they accounted for properly?

o Are the costs of the contributions as reported reasonable for the community inputs?

o Were there additional community contributions not reported?

o Assess whether contractors were ever paid for the part of works carried out with community contributions.

• Were community contributions an important factor in determining the cost effectiveness of NCDDP sub-projects relative to similar sub-projects supported by others?

• How reasonable are the costs for materials, transport, labor and other inputs?

• Were SPs designed to maximize community benefits through employment of local labor, procurement of local materials, or other means?

• Were SP designs and specifications selected to maximize value for money? Would other designs, technologies or methods have provided greater value?

2.3 Compliance with environmental and social safeguards under the NCDDP investments

The SPs visited were also assessed in regards to their compliance with the Project’s environmental and social safeguard standards and policies.

The scope of this assessment is as follows:

• Proper documentation and recording of Environmental Code of Practice (ECOP) and the Safeguard Checklist, and the verification and monitoring by the Township NCDDP office of contractor/community compliance with ECOP.

• Loss of land or private assets, the scale of impact, whether or not they are addressed through voluntary donations and if so, whether all conditions of voluntary donations as provided in the operations manual are met.

• Verification of whether any adverse environmental impacts occurred at the sub-project site, and how they were mitigated.

2.4 Operation and Maintenance (O&M)/Sustainability

The physical examination of the SP during the technical evaluation, for 2.1 above, also allowed for an accurate appraisal of the current state of O&M of the infrastructure. Additional information was gathered during the village implementation committee interviews.

The scope of questions to be answered are as follows:

 Is the current condition of SP infrastructure good, fair or poor?

 Have any major repair or restorative maintenance/ rehabilitation works conducted since the completion of civil works or does the current condition require such works? If so, what are the causes of defects? Break down the causes of defects into environmental/ natural factors; technical defects in design, implementation or materials.

 If any O&M works have been done, who did what O&M work?

 Was any routine maintenance (wear and tear and/or replacement of consumables) carried out on the sub-project?

 Is the quality of the O&M Plan sufficient? In particular, does it address both normal wear and tear, routine maintenance and replacement of spare parts, and reactive maintenance/ capital repair? Do the O&M plans adequately cover the requirements over 3 – 5 years of operation, and clearly spell out specific works to be done, and agencies responsible for and expected cost of respective works?

 Is there an O&M committee in place and functioning? What are the O&M arrangements? What are the roles and responsibilities (both financial and technical) of local governments/ line agencies and communities? Are roles and responsibilities separated for direct beneficiaries/ users and indirect beneficiaries?

 Was any training provided to communities on O&M (including refresher training), and if so, what types of training were provided? Did communities request and/ or receive technical support from local governments/ line agencies on O&M?

 Is an O&M fund in operation? Who holds the funds? What is the current value of these funds? How are contributions made? By whom? Are those expected to contribute able and happy to contribute?

 Was the O&M fund developed based on the consideration of technical requirements?

 Assess whether applicable user fees are affordable to users and sustainable to finance longer term O&M. Did the line ministries contribute to O&M expenses?

 Were necessary Government inputs (e.g., teachers and learning materials for schools, or health workers, drugs and equipment for dispensaries) provided adequately and in a timely manner?

 Are responsibilities, both financial and technical, clearly spelled out for community members and for the government?

 Did the community or contractor implementation modalities have any impact on O&M? What investment types are more suitable to community force account in terms of long-term cost effectiveness? What conditions have to be met to make the model of community force account cost effective in the long run?

 Does community capacity development carried out by the NCDDP contribute to SP sustainability cost-effectively? Compare the total cost including the cost of community engagement and capacity development of investments financed by different sources, taking into account (i) the current conditions of infrastructure; (ii) initial condition of infrastructure after completion and (iii) O&M works done. Any indication that the NCDDP’s investments in the capacity development of communities contribute to long-term sustainability of subprojects? If such an indication is observed, how cost effective is the NCDDP community capacity development in long-term sustainability of infrastructure?

2.5 Best Practices/Recommendations/Lessons Learned

The technical evaluation team members were encouraged to make note of particular instances where good or bad practices were observed or related during interviews. The field instruments provided areas where this commentary could be noted.

The main ideas for capture in this section are as follows:

 What examples of good practice can be drawn to enhance technical design quality, operation and maintenance and sustainability for future NCDDP SPs?

 What good practices can effectively address threats to sustainability?

 What are the key lessons learned from the sub-projects undertaken? What practices should be replicated and/or avoided in future sub-projects?

 Provide a list of key recommendations based on these good practices and lessons learned for the future design, implementation and maintenance of future NCDDP SPs

All questions from above are repeated within the reporting sections.

Recommendations of the Technical Evaluation are presented throughout the text of this report, associated with each item under discussion/analysis, and gathered together in Annex 1 for convenience.

 

3 Technical Evaluation Team Members

The technical evaluation and audit was led by Neil Neate, P.Eng. Neil was assisted by two Myanmar consulting engineers and eight engineers who are employed by DRD but those engineers are not employed by NCDDP.

The other technical evaluation (TE) team members and their professional qualifications are as follows: Aung Myo Nay, civil engineer; Nyein Thant, civil engineer; Nay Myo Min, civil engineer; Nay Zar Lwin, civil engineer; Thant Zin Tun, civil engineer; Han Myint Tun, mechanical engineer; Kyaw Min Naung, civil engineer; Saw Evans, mechanical engineer; Mya Thet Htwe, civil engineer; Min Min Than, civil engineer.

 

4 Sub-Project Sampling and Selection Process

4.1 Sampling Framework

Sub-projects were selected for the technical audit using a stratified random sampling methodology. A total of 160 Year 2 SPs were targeted as being 11% of the total population. A further 40 SPs constructed in Year 1 were to be examined with special emphasis on the Operation and Maintenance aspects of those SPs that had been in use for a year or more (in Kanpetlet, Kyunsu and Namhsan). The teams’ sampling in the Townships did not achieve this total for the Year 1 SPs due to security concerns in one Township and a misunderstanding in another. In the end, thirty (30) SPs were evaluated from Year 1, representing 8.5% of Year 1 SPs in the 3 Townships.

In order to do the sampling, all SPs were classified as a Main Type: Road, Water Supply, School, Electricity, Community Building, Bridge, Hydro Power, Jetty, and Sanitation. The distribution of these Main Types was determined across all project Townships for Year 2 SP. A weighting was applied in recognition of the fact that a 10% sampling and technical appraisal of Year 1 constructions has been completed in the original three project Townships. The final number of Year 2 SPs to be evaluated was determined from the weighted percentage applied against the desired sample size.

Following is a table showing these calculations.

Table 4.1 – Sampling Framework for Year 2 SP

SP Main Type Actual Distribution Weighted Distribution Number of SP to be Evaluated

Road 36% 30% 48

Water Supply 22% 20% 32

School 22% 20% 32

Electricity 6% 10% 16

Community Building 6% 11% 18

Bridge 4% 5% 8

Hydro Power 2% 2% 3

Jetty 1% 1% 2

Sanitation 1% 1% 2

Total 100% 100% 160

The additional Year 1 SPs to be evaluated were not selected in a similar manner. Year 1 SPs were identified from their coincident location within a village that had been selected for the Year 2 SP evaluations, in the three original Townships Kanpetlet, Kyunsu and Namhsan.

An additional criterion used in the selection process was the degree of remoteness for subject villages. Township project teams provided the distances from the Township to each SP village, which allowed the evaluation team to select SPs based on remoteness. Three degrees of remoteness were used during the preliminary selection of village SP sites: Not Remote (within five kilometres of a Township center), Remote (five to twenty kilometres) and Very Remote (over 20 kilometres). These distances were used to roughly classify the villages for the selection process, however final noting of a village’s remoteness was not done until the evaluation team was actually on the ground and evaluating a SP. Road conditions and other factors that influence relative remoteness could then be judged based on the actual experience of travelling to the subject village.

4.2 Sub-Project Selection Process at the Townships

The SP sampling framework, containing a list of the nine Townships and the desired number of SPs within each SP type, was transmitted to the NCDDP offices in each Township prior to the technical evaluation team’s attendance. This list prompted the local Township teams to start thinking about suitable SPs that could be visited and evaluated, bearing in mind the state of construction at sites, accessibility constraints, etc.

Upon arrival at a Township NCDDP office the evaluation team introduced themselves to the local staff and explained the evaluation’s purpose and proposed SP sampling methodologies. It was explained that the technical team wished to visit a random representative variety of SP, based on the NCDDP MIS distribution of infrastructure types within that Township. The local list of SPs was inspected with the sampling framework’s parameters in mind, and a preliminary list of suitable SP sites was created.

Often local officials would have one or more nearby SP sites in mind for this evaluation (based on the requirements set out in the SP sampling framework) and might have already contacted Village Project Support Committee (VPSC) members with a request for an inspection visit on that day. These SP technical inspection visits were often executed on the arrival day, in order to take advantage of the VPSC members who would be awaiting a visit.

Subsequent days’ travels and selected SP site visits were coordinated with Township Technical Assistance (TTA) staff and others at the Township office at least one or two days in advance in order that VPSC and O&M Sub-Committee members could be contacted, allowing them to arrange their affairs to attend the technical evaluation team’s visit. Technical Facilitator (TF) staff members were also able to prepare a folder of office paperwork and files to take along to the site visit.

4.3 Site Visit and VPSC Interview

The evaluation team communicated with the NCDDP TTA office and TF to indicate a time frame within which each particular SP site visit would take place. Travel times to more remote villages and an awareness that certain SP infrastructure types, such as water systems and lengthy road segments, might take longer were factored into the estimates of when the team would arrive at particular villages.

Technical evaluation teams would generally arrive at a SP village, meet the VPSC representatives, and then request to visit the SP site immediately. It is helpful to understand the location and state of the infrastructure before embarking on the evaluation questionnaire with the villagers. Questions can be phrased more appropriately and with less confusion if the evaluators have seen the infrastructure. Many questions regarding the SP can be asked in an informal way during the actual site inspection period. Evaluation teams consist of at least two individuals, which allow for a free-flow of questions and answers with a variety of people attending the site inspection. One inspector might choose to explore the attic of a school with one or more knowledgeable villagers following, while the other inspector can visit the toilet structure, for example, and make enquiries along different lines. In this way a technical evaluation can gather small pieces of intelligence from a variety of participant in different locations. Answers to questions can be compared by the inspectors and inconsistencies identified. Further lines of questioning can follow during the more formal, generally sit-down, sessions later on.

 

5 Technical Evaluation Methodologies

5.1 NCDDP Sub-Project Main Types vs. Technical Evaluation SP Types

During preparations for the technical evaluation it was noted that some types of rural infrastructure are represented in more than one Main Type, as described above in Section 4.1. For example, building construction is featured in Table 4.1 as schools, community buildings and sanitation structures. It was also observed that electrical infrastructure is represented in both Electricity and Hydro Power. These groupings would create reporting and coding difficulties for the technical evaluation data as it is collected, digitized, saved and analyzed.

SPs were therefore divided into SP Types, allowing each SP evaluated to be assigned a sub-project type code (alpha). The sub-project types identified for the NCDDP menu are as follows:

Table 5.1 – NCDDP Technical Evaluation 2015 Sub-project types

Type Sub-Project Type Descriptor SP Main Types Represented

A Building School, Community Building, Sanitation

B Bridge Bridge

C Water Supply Water Supply

D Road Road

E Irrigation Note 1

F Electricity Electricity, Hydro Power

Note 1: Although irrigation is on the menu for NCDDP, no irrigation SPs were evaluated during this study.

The analysis within this report is based upon the above sub-project types, and the findings for each specific sub-project type apply across all Main Types in which such infrastructure is found. For example, the technical evaluation’s conclusions regarding reinforced concrete practices will apply equally to buildings found in many SP Type A above, to concrete bridges and road structures in Types B and D, to concrete reservoirs in Type C, and to concrete channels in Type E, etc. Thus this evaluation’s findings for each sub-project type should be viewed and applied with equal interest across the NCDDP categories featuring such infrastructure.

5.2 Technical Evaluation Field Instruments

The technical evaluation (TE) teams used unique field instruments for each SP type. These field instruments consisted of a set of five checklists that were completed at each village where the subject SP was located. The topics of the Field Tools were as follows: 1 Technical Evaluation of Infrastructure; 2 Cost Effectiveness; 3 Environmental and Social Safeguards; 4 Operation and Maintenance/Sustainability; and 5 Key Issues.

The Field Tools are similar to those used in other countries, but were customized to suit the unique conditions of NCDDP’s work in Myanmar. This customization task was done in consultation with the NCDDP and WB during the first week of the assignment. Draft copies of the proposed instruments were submitted for comment. Valuable advice was received and the Field Tools were finalized for testing and training purposes.

Prototype sets of Field Tools were field tested in Tatkon Township, Naypyidaw Union Territory at several SP sites: a rural road upgrade, an electrical-grid extension scheme and a pumped water supply system. The Myanmar field team engineers were trained at this time also, with the full team visiting SPs of each type in order to use and understand each of the different SP type Field Tools. Subsequent feedback from the team spurred some minor changes to be made to the various checklists. Training sessions for the field team engineers continued in Laymyetnar, Kyunsu and Pinlebu Townships.

A sample of one of the Field Tools is attached to this report in Annex 2 – Building Technical Evaluation Field Instrument.

The technical instruments contain data fields that were filled in with a checkmark or notation at the SP site itself. Other parts of the field instrument would often be completed afterwards, during meetings at a village hall or VPSC member’s home. Following is a general summary of the data fields in each of the individual Field Tools:

Field Tool 1 – Technical Evaluation of Infrastructure – This three-page checklist is unique to each SP type. The five SP types were divided into a number of components, each to be rated separately (the rating system is defined below in Section 5.3). Components for the sub-project type Building, for example, started at the base: Foundation, Ground Beam, Wall, Column, etc., proceeding up to the Roof Structure. Where a particular component had several distinct aspects that should be evaluated separately, the component was subdivided, for example: Ring Beam – Reinforcement, and Ring Beam – Dimension. A complete list of each SP types’ components and aspects is provided in Annex 3.

This instrument also collected other SP quality ratings (Overall Quality, Design Completeness, SP Functionality, etc.) that are more fully discussed in Section 5.4 below. Space is provided on all the checklists for comments to be written.

Some parts of this field instrument were also completed for the Comparable Sub-Projects (CSP) visited, making possible a comparison of NCDDP’s SP technical quality with those of other organizations.

Field Tool 2 – Cost Effectiveness – This checklist consists of two pages. The first page contains data fields unique to each SP type. TE team members were instructed to examine construction plans, as-builts and specifications to verify and record the dimensions and materials of the SP. Information from the first page was used to determine each SP’s basic unit costs, allowing comparisons to be made between SPs. Where possible and when time allowed, teams would check some of the measurements at the SP site. The second page of this instrument was the same for all SP types, and required that SP accounting records be studied to determine if any special costs had been incurred, for specialized trades or for transport of materials, etc. Community contributions to the SPs were also noted. This field instrument was also completed for all CSP visited, making possible a comparison of NCDDP SP unit costs with those of other organizations.

Field Tool 3 – Environmental and Social Safeguards – This single page checklist is common to all SP types. TE teams examined the village SP implementation files to verify the inclusion of all required policy and code of practice documents, as well as records of monitoring by NCDDP staff. A physical inspection of the SP was also performed to visually confirm the completion of requirements as set out in ECOP.

Field Tool 4 – Operation and Maintenance/Sustainability – This checklist is comprised of two pages. The first page contains data fields unique to each SP type. The second page collects standard information from O&M Committee members at SP sites and requires the team to examine SP documentation and make notes from each O&M Plan.

Field Tool 5 – Key Issues – The checklist for this page is unique to each SP type. It contains a variety of common problems or issues that typically are found in rural infrastructures. The Building SP Key Issues list, for example, contains a checklist for the following visible problems: Inadequate overlap of roof sheeting; Improper connection of roof to truss; Unreinforced, inadequate, or improperly located splices in truss members; Missing steel strapping in truss; etc. The identification of these issues augments and contributes to the understanding of ratings assigned in Field Tool 1. The number of issues available for each SP type are as follows: Building 37 items, Bridge 25, Water Supply 28, Road 20, and Electricity 9.

5.3 Technical Rating System

Using Field Tool 1, each component or aspect of the SP infrastructure was rated as being one of five choices: Meets Spec (Specification); Slightly Below Spec; Below Spec; Not Inspected; and Not Applicable. The component or aspect was examined in its current condition and reasonable allowances were made for normal wear-and-tear and degradation.

These ratings are defined for this technical evaluation as follows:

• Meets Specification (Meets Spec) – The sub-project component or aspect meets the plans, specifications, or criteria as set out in the Sub-Project Proposal.

• Slightly Below Specification (Slightly Below Spec) – The sub-project component or aspect displays certain characteristics that could be improved upon within its design, materials, construction, operation/maintenance or environmental conditions to meet the plans, specifications or criteria presented in the Sub-Project Proposal. This rating will normally be accompanied by written commentary describing improvements that can be made to improve technical quality and sustainability.

• Below Specification (Below Spec) – The sub-project component or aspect was either (i) not constructed according to the approved plans or specifications in the Sub-Project Proposal, or (ii) presents a clear and present danger to the life or safety of users. This rating will normally be accompanied by written commentary describing improvements that must be made to ensure technical quality and sustainability.

• Not Inspected – It may occasionally be impossible for the TE team to inspect a certain aspect of a sub-project. For example, many completed buildings feature ceilings with limited or no access to the attic. TE teams may not be able to inspect the interior of a building’s roof structure in these instances. The TE team will question the village and Township personnel in this instance to verify sub-project details as much as possible.

• Not Applicable – Some components or aspects will not be applicable to sub-projects. For example, the component Ceiling is included in the Building Checklist, but many building sub-projects do not include such installations.

Evaluators take into account normal deterioration of components over time. The use of this rating system assumes that standard O&M tasks have been carried out. O&M is rated separately for all SP type, and if it has not been carried out properly, the O&M SP component would be rated Slightly Below or Below Spec according to conditions. Extreme degradation due to poor O&M is not the infrastructure’s fault (where the SP works were well designed and installed).

5.4 Quality Ratings and Other Criteria

The second page of Field Tool 1 offers the evaluator an opportunity to rate the SP’s construction quality as well as several more general and less-technical areas. These “Overall Project Assessment” categories are as follows:

• Sub-project construction quality (rated in accordance with WB six-point rating system);

• Design completeness (Good, Fair, Poor), with opportunity to write a comment;

• Design consultations with users (Yes or No), with opportunity to write a comment;

• SP Proposal documentation check. (Rated Yes if documentation found, No if not.).

These quality ratings are defined and further discussed below in Section 7, in separate sections for each. Analysis of the sub-project quality ratings gathered in this part of Field Tool 1 is presented along with commentary.

A full listing of the 210 NCDDP SP evaluated and their WB quality ratings (first bullet above) is provided in Annex 4.

The second page of Field Tool 1 also provides space for the evaluator to write a brief sub-project description and add comments regarding particular issues that were noted during the evaluation, as well as Lessons Learned at each site. All of this commentary was digitized and submitted with the field data. A summary of the comments from each SP is provided in Annex 5.

5.5 Field Checklist Data Input

The field data from Field Tools 1 to 5 was input to digital spreadsheets in the office after the fieldwork was complete. The digital data input spreadsheets are patterned after each of the hard-copy Field Tools. These forms allow input of the field information in a format very similar to that in which it was gathered, thereby reducing input errors.

The Myanmar consultant/DRD team members input the field data for each sub-project evaluated and saved these spreadsheets to computer files using standard naming formats. The naming formats are based upon the national administrative numbers for each State/Division, Township, Village Tract and village along with an added code for Sub-project Type to enable the sorting and correlation activities to take place.

Thus, the file naming system used for this technical evaluation is as follows, substituting numbers for each square-bracket item: [State/Division]-[Township]-[Village Tract]-[Village]-[Sub-project Type], where GoM administrative numbers are used along with the SP type codes per Table 5.1 above. Thus, a water supply SP in Saw Chaung village, Kant Thar Yone Village Tract, Kanpetlet Township, Chin State is represented in the digital analysis as 04 08 01 02 C.

The digital spreadsheets allow the data to be systematically filed, grouped and analyzed using computer sorting techniques. The data within the sub-project spreadsheets can, for example, be sorted by year of construction, by location, or by the rating evaluations under Overall Construction Quality or Technical Facilitation. Sorting procedures can be used to reveal trends or to highlight problem areas.

6 NCDDP Sub-Projects Evaluated

6.1 Sub-Project Sample

A complete list of the NCDDP SP sample and the comparable SPs is provided in Annex 4.

6.2 Sub-Project Sampling Criteria vs. Audit Sample

• Geographical distribution of sub-projects by two different regions in the country (Northern and Southern)

– The sampling framework distributed the evaluated SPs amongst the nine Townships according to the statistical percentages of Main Types within each of them. The following table provides this theoretical distribution alongside the final sampling numbers in each Township.

Table 6.2.1 – Geographical Distribution of Sub-Projects

Statistical Distribution of Year 2 SP (Number of SP) June 2016 Technical Evaluation

Kanpetlet 11 11

Pinlebu 27 28

Kyunsu 17 20

Sidoktaya 15 19

Ann 16 16

Htantabin 17 22

Namhsan 14 13

Laymyetnar 28 29

Tatkon 15 22

Total 160 180

• Proportional representation of sub-projects by type

– The selection of Year 2 SPs in the Townships met or exceeded the suggested sampling framework, as shown in the following table.

Table 6.2.1 – Year 2 NCDDP Sub-Project Sampling Criteria vs. Numbers Evaluated

Building Bridge WS Road Electricity Total

Criteria 53 8 32 48 19 160

Evaluated 55 13 32 61 19 180

– Year 1 SPs were sampled as available in or near Year 2 villages without regard to SP type.

• Projects implemented under community, contractor and joint implementation modalities

– The TE teams did not select SP sites to visit based upon construction modalities. It was felt that applying this criteria would unnecessarily complicate the selection process, and that random sampling by SP type within the Townships would produce an adequate representation of all construction modalities. Contractor modality ranges from 0% in several Townships to 74% in Kanpetlet.

Table 6.2.1 – NCDDP Sub-project Construction Modality

Community Force Account Contractor Joint Modality

NCDDP MIS information (% of SP) Approx. 72% Approx. 15% Approx. 13%

2015 Tech. Eval. (number of SP) 184 8 18

Study (%) 87% 4% 9%

– The percentage of CFA in the table above may be lower than as shown. Some double-checking of the technical evaluators’ data input sheets reveal some mis-coding taking place on this item. The TE teams may have been unclear on the construction modality question and neglected to register some contractor or joint modality SPs.

• Location of project sites (remote areas will form at least 50% of the sample with the remaining from the non-remote areas)

– The degree of remoteness for each SP was rated as Not Remote, Remote, Very Remote and Extremely Remote.

– The SPs evaluated were judged as follows: 26% Not Remote, 41% Remote, and 28% Very Remote and 4% Extremely Remote.

• The operation and maintenance status of 40 Year 1 SPs was to be evaluated in Townships Kanpetlet, Namhsan and Kyunsu, taking advantage of visits to Year 2 SPs in these Townships.

– Communication difficulties with the technical evaluation teams caused a misunderstanding of the intention to evaluate O&M aspects of Year 1 SPs. Only 30 Year 1 SPs (approximately 8.5% of Year 1 SPs) were visited during this evaluation. The target was 40 Year 1 SPs, to augment the 10% sampling by Garvan O’Keefe in Sept. 2014.

7 Technical Design Quality – Findings

Following are the questions to be answered from the Terms of Reference and scope, with discussion and analysis presented for each item as appropriate.

7.1 What is the technical quality of the design? Assess the as built condition in as far as possible as good, fair or poor based on list of key criteria developed for each major type of subproject to be checked as basis for the technical quality assessment.

Technical Quality of Design

Village SP files were studied to verify that the appropriate documentation was present and properly completed. The VPSC was also questioned regarding the liaison that the NCDDP provided during the design and construction period.

Proper design drawings created by NCDDP staff, checked by qualified engineers and provided to the construction site are vital to properly executed SPs. The technical quality of the designs was rated by the TE team using Field Tool 1, under the item Design Completeness, which included a general appraisal of the construction documentation, design drawings and details of construction, and specification requirements. Table 7.1.1, below, shows how an aggregate of each Township’s files were rated, along with the total for all SP evaluated. On the following page is a chart showing a visual representation of this table’s results.

Table 7.1.1 – Technical Quality of Design (aggregate all SP evaluated) by Township

Good Fair Poor

Number % Number % Number %

Kanpetlet 5 20 20 80

Pinlebu 21 75 7 25

Kyunsu 6 20 23 77 1 3

Sidoktaya 12 63 7 27

Ann 5 31 10 63 1 6

Htantabin 3 14 19 86

Namhsan 1 5 16 84 2 11

Laymyetnar 16 55 12 42 1 3

Tatkon 3 14 18 86

Average (210 NCDDP sub-projects evaluated) 71 34 134 64 4 2

.

Chart 7.1.1 – Technical Quality of Design (aggregate all SP evaluated) by Township

The following table, 7.1.2, examines this same data according to SP type.

Table 7.1.2 – Technical Quality of Design by SP Type (% of SP Evaluated)

% Building

(64 SP evaluated) Bridge

(15) Water Supply (36) Road

(72) Electricity (23)

Good 40 20 42 29 27

Fair 58 80 58 68 68

Poor 2 3 5

Discussion:

In the Table 7.1.1 above, several Townships have a majority of SP designs rated Good (Pinlebu, Sidoktaya and Laymyetnar). The remaining Townships have a majority of their files rated Fair for completeness, indicating that some improvements can likely be made. Those with village SP design files rated Poor (particularly Ann and Namhsan) might choose to focus on improving this overall quality indicator with a concentrated effort to increase the quality of SP designs and documentation.

Table 7.1.2 provides an indication of where such improvements might best be directed. In this table it can be seen that road and electricity SPs have several designs that were considered Poor and also tend toward Fair ratings. Building SPs are somewhat better with a larger percentage of Good ratings although there was one poorly rated SP (in Laymyetnar). Water supply SPs had a majority rated Fair while Bridge SPs are of Fair quality 80% of the time.

Recommendation 1: NCDDP should convene a technical sharing session where Township and Union engineering representatives meet to exchange ideas on how SP designs and file documentation can be improved, presenting examples.

As-Built Condition Assessment

Field Tool 1 allowed each component or aspect of the individual sub-project types to be rated as being one of five choices: Meets Spec. (Specification); Slightly Below Spec; Below Spec; Not Inspected; and Not Applicable. The rating is a reflection of how the component/aspect has followed the SP specifications, the quality of its material composition/inputs, and its consistency with the bill of quantities (BoQ). Critical design elements such as toilet facilities, if dropped from a SP, would merit a Below Spec rating and, likely, a specific written comment on the field tool. The rating system of Meets Spec/Slightly Below Spec/Below Spec is analogous to Good/Fair/Poor.

The technical quality ratings can be viewed in detail for each component of each SP, to understand how well each particular piece of the infrastructure has been constructed. If one is examining the data collected for a bridge SP, for example, the individual technical quality ratings for 14 different components can be reviewed, from Layout and Foundation to Connections and Apron/Ramp. A detailed examination of the data from one bridge might reveal that the concrete foundation and reinforced column works were done poorly, while the upper wood assembly was done in a very good and proper fashion. This might show that local unskilled workers did not receive adequate direction while performing the underside concrete support works but grew in confidence when they were working with local timber and wood-joinery techniques. Notations to each individual SP data input sheet might be informative in regards to the particular circumstances at an individual SP.

The examination of each individual SP’s ratings, while interesting, is obviously time-consuming and of small worth if one is seeking to identify broad-based ideas that will improve NCDDP’s construction program. It is possible to aggregate the component ratings, so that one can identify general trends in the data gathered. In regards to water supply SPs, for example, the ratings recorded for each of 16 components/ aspects can be gathered and examined as a representation of the average quality rating of each component/aspect of water supply SPs as a whole. It can be argued that an aggregate of the ratings from representative samples will provide insights into the whole group of SP types, and will point towards those parts of NCDDP’s construction methodologies that most require improvements. The following table presents the aggregate of ratings from 36 NCDDP water supply SPs. Table 7.1.3 presents an abbreviated list of water supply components/aspects. A full list of the components/aspects rated for each SP type is provided in Annex 3.

Table 7.1.3 – Aggregated Sub-Project Ratings for Water Supply Components, % of SPs Evaluated

Meets Spec. Slightly Below Spec. Below Spec.

1 Watershed Protection (29 SP evaluated) 66% 28% 7%

2 Water System Design (30) 70% 27% 3%

3 Transmission/Distribution Pipe (18) 44% 56%

4 Reservoir – Easy to Clean (28) 89% 11%

5 Public Tapstands – Drainage (11) 45% 36% 18%

6 Water Pressure and Quantity (17) 71% 29%

Discussion:

The table above provides detailed data in regards to specific parts of typical water supply SPs. For example, in line 1, gravity-fed systems usually depend upon a watershed area for the supply of pure water. It is also important that boreholes are located at least 100 feet from a source of contamination. Having some form of protection for these areas is a sensible idea. This can take the form of fencing and making the watershed out-of-bounds or monitoring activities and ensuring that nothing deleterious is released. Of the 29 SPs that were assessed in this regard, 66% (19 SPs) were considered to Meet Spec. A further 28% (8 SP) were rated Slightly Below – these may have been gravity systems where it was apparent that some uncontrolled activities might be taking place above a source or a borehole SP installed without proper fencing, toilets or pooled water nearby, etc. Two SPs (7%) were considered to be Below Spec. Commentary was recorded for these two SPs: in the village Zee Phyu Kone, Sidoktaya, it was observed that waste water is able to flow into an unfenced intake on the Mon River; and in village Kyauk Hpyu Ywar Ma, Namhsan, two intakes on a stream have not been protected with fencing, allowing animals access to the water supply area.

Other components of water systems can be examined in similar manner: for the most part, designs are good (line 2, 70% Meets Spec) although there is room for improvement (standard designs are not often used and require additional details to be added when they are used); transmission and distribution piping (line 3) appears to require some analysis and possible changes to the current methodologies (56% Slightly Below Spec), likely due to unburied or improperly supported pipes; reservoir designers have been successful, ensuring that 89% of SP reservoirs are equipped with proper drains to make them ‘easy to clean’ (line 4); public tapstands have not been formed correctly in a majority of cases (line 5, 55% Slightly Below or Below Spec), allowing water to pool and become a nuisance and potential disease vector breeding area; and finally, water pressure and quantity at a majority of SPs (71%) was judged as suitable.

A thorough analysis of all of the components/aspects for each of the SP types will be offered in Section 11 – Best Practices and Recommendations.

In order to understand the technical quality of the full breadth of SP works, however, all of the ratings assigned the components/aspects of all the SPs can be aggregated, providing a useful overview of NCDDP’s entire construction program.

An analysis of these ratings shows that, when considering an aggregate of all sub-project components, 78% of the sub-projects have been constructed in accordance with the plans and specifications contained in the Sub-Project Proposals and considered to Meet Specification, with a further 19% rated Slightly Below in terms of meeting the intent of the sub-project proposal. Only 3% of technical ratings were Below Specification.

The chart below represents this finding, using an aggregate of all of the technical components of the sub-projects evaluated.

Chart 7.1.1: Technical Quality Rating of Sub-Project Construction

The following table presents separate totals for each of the sub-project types evaluated.

 

Table 7.1.4: Summary of Component Technical Ratings by Sub-project Type

Meets Spec. Slightly Below Spec. Below Spec.

Building (64 sub-projects evaluated) 79% 19% 2%

Bridge (15) 91% 6% 3%

Water Supply (36) 79% 18% 3%

Road (72) 75% 16% 9%

Electricity (23) 57% 42% 2%

Average (210 NCDDP sub-projects) 78% 19% 3%

Discussion:

The bridge SPs evaluated were found to most consistently Meet Spec: 91% of bridge components were judged to satisfy the Sub-project Proposal, technical specifications and Bill of Quantities.

Buildings and water supply SPS were both rated at 79% Meets Spec, with a further 19% and 18% respectively deemed Slightly Below Spec. Only 2% and 3% respectively were rated as being Below Spec.

Road SPs were rated as meeting specification 75% of the time, with an additional 16% being considered Slightly Below Spec. Of concern, however, is the 9% of road components that were considered to be Below Spec. Aspects highlighted here are: improper road cross-sections (crown/camber) in Kanpetlet, Sidoktaya, Htantabin, Namhsan and Tatkon; inadequate roadside ditching mainly in Sidoktaya; unstable slopes below the road in Htantabin, Laymyetnar and Pinlebu. Section 11.4 offers more detail in regards to these issues.

Finally, only 57% of electrical components evaluated were rated as Meeting Spec, with 42% being considered Slightly Below and 2% Below Spec. This result must be examined in detail because of the danger to users that might be presented with improper electrical installations. Section 11.5 deals with these issues in depth.

A detailed examination of the data gathered for each component will give an understanding of how each of these particular SP types are falling below specification. A full analysis of this data is presented in Section 11, Best Practices and Recommendations. Strategies to improve construction materials or methodologies will be offered.

A similar examination of the data can be done for all SP evaluated in each Township, as shown in the following table.

 

Table 7.1.5: Summary of Component Technical Ratings by Township

Meets Spec Slightly Below Spec Below Spec

Kanpetlet 65% 30% 5%

Pinlebu 79% 18% 3%

Kyunsu 74% 25% 1%

Sidoktaya 72% 17% 11%

Ann 62% 37% 1%

Htantabin 77% 10% 13%

Namhsan 85% 10% 5%

Laymyetnar 81% 14% 4%

Tatkon 81% 11% 8%

Average (210 NCDDP sub-projects) 78% 19% 3%

Discussion:

Here it can be seen that Ann and Kanpetlet have some challenges with their delivery of properly designed and/or constructed infrastructures. The aggregate total of all components of these Townships’ SPs that Meet Spec is 65% and 62% respectively, far less than the national average of 78%. The reasons for these apparent problems in Ann and Kanpetlet are not immediately discernable. The technical evaluators did not gather data on numbers of staff in each Township, nor their training or other possible causes of these technical failings. Frequency of technical facilitator visits to each SP site was recorded, with Ann and Kanpetlet registering an average of 6 and 4 visits respectively (against a national average of 5 technical facilitation visits/construction period). A detailed study of the actual ratings for each type of infrastructure in each Township may allow some conclusions to be made about where Ann and Kanpetlet’s problems lie. Both of these Townships display infrastructure with suitably low percentages of components Below Spec (1% and 5%, respectively, against a national average of 3%).

Pinlebu, Kyunsu, Sidoktaya and Htantabin are performing satisfactorily, with some issues to address before several of these Townships can move upward in quality to the national average and higher. Sidoktaya and Htantabin’s dominant problems (note their higher Below Spec percentages in the table above) are numerous road SPs where the cross-section, inadequate roadside ditching and unstable slopes were considered poorly done. These issues should be relatively easy for the technical facilitation teams to improve upon in future cycles.

Laymyetnar, Tatkon and Namhsan demonstrate good practices for the most part. Some SP types in these Townships do have issues that should also be investigated and improved upon during the next cycle.

Following is a breakdown by SP type for each of the Townships. Each chart represents the percentage of components/aspects that meet specifications for each SP type within each Township.

Charts 7.1.2: All Components/Aspects Aggregated, by SP Type

The charts above depict the aggregated ratings for each of the SP types within each Township. NCDDP management and field personnel can examine these charts to see where improvements must be made. For example, two charts above, it can seen that Laymyetnar electrical SPs have a large percentage of components rated Slightly Below. The field tools provided the technical evaluators with detailed breakdowns of each SP type (into components and aspects), so that identification of the reasons behind these ratings will be possible. Section 11 of this report thoroughly explores and explains the evaluators’ rationale for giving such ratings.

7.2 What is the quality of materials/inputs and are these consistent with the BOQ in the bidding documents?

The quality of the inputs to each SP and their consistency with BoQ and specifications was assessed as part of the technical rating provided in Section 7.1, reported above. Where quality of inputs was perceived to be problematic for a SP, the rating assigned would be Slightly Below Spec or Below Spec.

An analysis of these findings, broken down by SP type and by component is fully presented in Section 11, Best Practices and Recommendations.

7.3 Did the sub-projects follow the technical specifications as designed? Were any critical design elements, such as latrines, dropped?

Similar to 7.2, above, SPs were rated based on the technical specifications presented within the SP documentation. Should a SP have not followed the scope as outlined in the village documentation, a rating of Slightly Below or Below Spec would have been assigned as appropriate. The omission of critical design elements would normally spur a rating of Below Spec (and accompanied by a written comment of explanation).

Table 7.1.4, Summary of Technical Ratings by SP Type, on page 26, shows that an average of only 3% of the components making up all SP types were considered to be Below Spec. (with a heavy weighting of these associated with road SPs). Considering that building SPs display an average of only 2% Below Spec, it is likely that very few (if any) critical design elements, such as latrines, are omitted from site works. It might be argued, however, that the high incidence of Below Spec components in road SPs is, indeed, due to the absence of critically important elements, for example drainage works or steep slope retention. A close examination of the data from the field, paying special attention to those SPs with Below Spec ratings, is warranted.

Detailed analysis of these data and associated findings are provided in Section 11.

7.4 What documentation exists to show that the sub-project meets the design and specification requirements; and

Have all technical requirements been met and defects addressed before sub-projects are handed over to communities?

A number of design, file documentation, construction, hand-over and procedural indicators were verified and checked by the TE team at each SP site visited. The results of which are in the following table.

Table 7.4.1 – Design Process and Construction Documentation (Aggregate of all SP evaluated)

Design and Documentation Criteria Yes No

1 End users consulted during the design process 99% 1%

2 As-built records in possession of Village Project Support Committee 57% 43%

Good Fair Poor

3 Final Inspection form and SP file completeness 23% 69% 7%

Discussion:

NCDDP consultation with the village user groups during the design stage is an important part of the implementation process. As can be seen here, all VPSC (except for one) reported that NCDDP staff spent time with them as the SP planning and design was underway. This result is a strong indication that these village interactions is standard NCDDP practice, something that deserves commendation.

A marginal majority of VPSC files contained as-built record drawings, with room for improvement. The development of as-built record drawings is one of the ways that NCDDP engineers and planners can learn from their ongoing works. Changes to SPs that are spurred by user suggestions or field conditions should be noted during the as-built revision process. As-built record drawings are an important part of the handing-over process. These drawings show any changes that were made to the design or construction of the infrastructure. Often designers and engineers can learn from these changes to make future SP designs better and more useful to villagers.

Recommendation 2 – NCDDP engineers and technical staff must check that properly executed as-built drawings are created for all completed SPs. Design changes should be reviewed with senior personnel so that standard template drawings can be altered if warranted.

The evaluation of SP file completeness on Line 3 shows that NCDDP and VPSC personnel are generally aware of the project’s requirements (notwithstanding the absence of adequate as-built drawings in many cases). The evaluators rated the SP as Good, Fair or Poor based upon the file contents and level to which each form had been completed. While 93% of the SP filings have been found to be Fair or better, there is obviously much room for improvement on this topic.

Recommendation 3 – NCDDP should find ways of improving the completeness of SP documentation at the village level. All forms must be completed with the full participation and understanding of the VPSC.

7.5 Did the sub-projects take into account DRM measures? If so, how?

Disaster risk management (DRM) has not yet been brought into the NCDDP SP planning and implementation process. Technical evaluators were asked to make note of situations where disaster risks seem apparent at SP sites. None were reported. The NCDDP does not have DRM tools at the present time. The ongoing development of roads, bridges, water systems, buildings and other infrastructure in remote areas will assuredly introduce DRM issues to construction sites. The introduction of DRM check-lists to the SP design process will help mitigate future problems.

Recommendation 4 – A DRM training course should be held to emphasize the responsibility of designers to more fully consider the forces of nature when planning rural infrastructures.

7.6 Sub-Project Overall Quality Ratings

The second page of Field Tool 1 provides a section where the evaluator, having evaluated the SP Proposal and each of the components of the infrastructure itself, can review the sub-project as a whole entity, taking into account the severity of imperfections or deficiencies in some aspects of the construction. These ratings are in conformance with the World Bank’s standard six-point table, as follows.

Table 7.6 – Overall Quality Rating system

1. Highly Satisfactory (HS) Project fully complies with or exceeds policy requirements.

2. Satisfactory (S) Minor shortcomings exist that do not have a material impact on compliance with policy requirements or achievement of development objectives and implementation progress.

3. Moderately Satisfactory (MS) Moderate shortcomings exist that do not have a material impact on compliance with policy requirements or achievement of development objectives and implementation progress.

4. Moderately Unsatisfactory (MU) Moderate shortcomings exist in compliance with policy requirements or achievement of development objectives and implementation progress but resolution is likely.

5. Unsatisfactory (U) Significant shortcomings exist in compliance with policy requirements or achievement of development objectives and implementation progress and resolution is uncertain.

6. Highly Unsatisfactory (HU) Major shortcomings exist in compliance with policy requirements or achievement of development objectives

Note: A complete listing of the SP evaluated and their individual quality ratings is provided in Annex 4.

Chart 7.6: Sub-Project Quality Rating

The infrastructure associated with nine (9) sub-projects was found to be Highly Satisfactory with another 186 SPs being rated as Satisfactory. Only 15 rural infrastructures were evaluated as Moderately Satisfactory or lower.

Most of the infrastructure examined during this evaluation was considered to be Satisfactory in its construction and documentation quality. A suitable number of sub-projects were rated as Highly Satisfactory.

7.7 Remoteness

The technical evaluation Field Tool 1 provided data fields where the evaluator could rate the degree of remoteness for a SP village. The initial SP sampling used three degrees of remoteness. A fourth degree, Extremely Remote, was added later. The degrees, their definitions, and number of SP for each are as follows:

Table 7.7.1 – Degrees of Remoteness and Sample Number of SP

Definition No. of SP

Not Remote Close to a main road and within 30 minutes drive from Township 55

Remote Off main road; within 2 hours of Township 87

Very Remote Between 2 and 4 hours from Township 59

Extremely Remote Greater than 4 hours from Township 9

The data were sorted to determine if a village’s degree of remoteness played a significant part in the technical quality rating of a sub-project’s components. A hypothesis might be that the technical quality of a sub-project will go down as the degree of remoteness goes up, due to a number of possible factors: increased difficulty for technical facilitators to visit the site; reduced number of skilled labourers being available; increased difficulty in securing proper construction materials; etc.

Table 7.7.2 – Remoteness, Aggregate of Ratings for All SPs, All Components

Meet Spec Slightly Below Below Spec

Not Remote (55 SP) 78% 21% 1%

Remote (87 SP) 78% 18% 5%

Very Remote (59 SP) 79% 19% 2%

Extremely Remote (9 SP) 75% 25% 0%

Discussion:

It can be seen that very little fluctuation of the aggregated sum of ratings is evident when comparing SPs’ degree of remoteness. This indicates that NCDDP technical facilitation efforts produce roughly equal results regardless of the SP villages’ remoteness. (Another glimpse into this aspect of findings is in Section 7.9 – Technical Facilitation.)

7.8 Construction Year

Spreadsheets were sorted to determine if there are any apparent trends in technical quality based upon when the SP was constructed. The main difference that might influence technical aspects of SPs according to cycle is the frequency and quality of technical facilitation and supervision (assuming that quality of material supply and local skilled labour remain the same). The influence of technical facilitation is studied in Section 7.9, below.

Table 7.8.1: Construction Year, Aggregate of Ratings for All SPs, All Components

Meets Spec Slightly Below Below Spec

Year 1 (30 SPs) 70% 29% 1%

Year 2 (180) 79% 18% 3%

The table shows that the overall technical quality of SPs has increased from 70% to 79% of components meeting specification. This may be an indication of an increasingly knowledgeable staff that is gaining experience.

7.9 Technical Facilitation

The TE teams gathered information regarding how often technical facilitators had attended SP sites. Check boxes were offered for Total Number of Visits; Number of Visits/Week; Number of Visits/Month; and Duration of Construction. TE team members always used the first box, Total No. of Visits, and consulted the VPSC’s construction records book to inspect comments left by technical facilitators during each visit.

Technical facilitation visits to SP sites averaged 5.9 inspections/construction period (from 199 SP that reported this information). This data was also sorted by remoteness in order to see if the number of inspections fluctuated according to this parameter.

Table 7.9.1 – Technical Facilitation Visits/Construction Period, by SP Remoteness

Not Remote Remote Very Remote Extremely Remote

No. of Visits 6.5 5.9 5.3 4.2

Discussion:

The data returned from the field has ably demonstrated that the more remote a SP is, the less frequent are its technical facilitation visits. It can be seen that the Extremely Remote sites are receiving 65% of the technical facilitation visits that are provided to Not Remote sites. Referring back to Section 7.7 above, however, one can see that the technical quality of more remote NCDDP infrastructure has not appeared to suffer greatly from this lesser amount of technical facilitation.

The frequency of technical facilitation can also be studied by Township and correlated with the first column of technical quality data from Table 7.1.5 above.

Table 7.9.2: Average Number of Technical Facilitation Visits by Township – Year 2

Average Number of Facilitation Visits/Construction Period Aggregate of All SP Components Meets Spec

Kanpetlet 3.6 65%

Pinlebu 7.9 79%

Kyunsu 4.5 74%

Sidoktaya 4.7 72%

Ann 5.6 62%

Htantabin 4.9 77%

Namhsan 3.5 85%

Laymyetnar 5.2 81%

Tatkon 8.6 81%

National Average 5.1 78%

Discussion:

There is some correlation between the Townships experiencing a lesser number of facilitation visits/(construction) period and a lesser overall technical quality, as well as the inverse (greater number of visits = higher technical quality). In the table above, almost all Townships with a lower than average number of visits/period (underlined) also display a lower than average aggregate technical quality, as well as the inverse. The correlation is not exact, however, with two exceptions: Namhsan has the lowest number of visits/period but the highest aggregate total of components that Meet Spec; and Ann with a higher than average visits/period has the lowest number of infrastructure components rated Meets Spec.

The technical evaluation teams did not collect the numbers of TF within each Township.

It is important to recognize that the percentages offered in table above are aggregated totals for large numbers of SP infrastructures of all types. Closer inspection of specific SP types and their individual components will reveal areas where improved or increased technical facilitation is required (Section 11 will examine this issue further).

Recommendation 5 – NCDDP should examine the human resources of each Township with respect to equalizing the number of technical facilitation visits to SP villages.

7.10 Universal Accessibility

Universal accessibility (UA) is the concept that public infrastructures and services should be designed and constructed to be inherently accessible to older people, people without disabilities, and people with disabilities.

The addition of UA facilities to public buildings can often be done for approximately 1% of the infrastructure’s total budget.

Recommendation 6: NCDDP should consider revising its engineering design guidelines to include explicit provisions for UA to public building infrastructure.

Recommendation 7: Ramps for the disabled are an important feature to guarantee Universal Accessibility to public infrastructure. Ramps should not be constructed steeper than 16% (wheelchair accessible with helper) and should feature a rough/non-slip surface. Ramps steeper than 5% should be equipped with a proper handrail.

8 Cost Effectiveness

The technical evaluation of NCDDP SP and Comparable Sub-projects used Field Tool 2 for gathering information that would aid in determining the cost effectiveness of the investments. The instrument provided numerous data fields for key infrastructure financial information, dimensions, materials and construction management costs. Technical evaluators examined SP/CSP file resources at the site, village and Township levels to complete these checklists, as well as physically measuring the rural infrastructures to confirm their as-built condition. The creation of spreadsheets containing all of this information has allowed comparisons to be made and conclusions drawn in regards to the cost effectiveness of investments in NCDDP SP versus those made in comparable infrastructures by others.

The Cost Effectiveness field tool was unique for each type of SP (Building, Bridge, Water Supply, Road and Electricity). The Building data sheet, for example, required length and width of the building, number of rooms, type of materials used, etc., while Water Supply required length and size of pipe, size of reservoirs, number of tapstands, etc. A portion of the field tool, pertaining to standard SP management costs, was common to all SP types.

Bills of Quantity, designs, specifications and other SP documents were examined in order to record relevant data for these comparisons. SP dimensions were checked at the sites in order to confirm both as-built drawings and unit area costs of construction.

Following is a table showing the number of NCDDP SP types versus the CSP evaluated by the technical evaluation teams.

Table 8.0.1 – NCDDP and Comparable Sub-Projects by Type

Building Bridge Water Supply Road Electricity

NCDDP SP 64 15 36 72 23

CSP 6 3 3 7 3

It can be seen that the field data gathered during this evaluation for Building and Road SP types will be more reliable than that collected for the other SP types, due to the smaller CSP sampling size accorded Bridge, Water Supply and Electricity. Comments below will reflect limitations that should be placed upon any analyses of these types.

Recommendation 8: Future audits should develop methodologies to increase the number of comparable SPs evaluated so that analysis and findings can be made with more certainty.

The comparable sub-projects evaluated during this study were as follows.

Table 8.0.2 – List of Comparable SP Agencies

SP Type Township Village Tract Village Year Agency

Building Kan pet let Hmuh chin ding Shin baung 2015 DRD

Building Kyun Su Taw Pyar Pan Zin 2015 Regional Budget

Building Kyun Su Than Doke Taw Nauk Le 2015 Hluttaw Fund

Building Kyun Su Ma Aing Lin Ma Lo 2015 Hluttaw Fund

Building Kyun Su Maung Hlaw Maung Hlaw (Lower) 2015 Regional Budget

Building Tatkone Kyee Chaung Kaunge Yar 2015 Ministry of Education

Bridge Htantabin Htein Hnit Pin Ywa Thit 2015 DRD

Bridge Laymyethnar Tha Khut

Chaung Kyun Chan Kone 2015 DRD

Bridge Tatkone Oh Shit Kone Naung Nga Pin(South) 2015 DRD

WS Pinlebu Man Ton Pay Kon 2015 DRD

WS Sidoktayar Kyaung Thaik Pa Zu 2015 DRD

WS Aing Tha Byu Aing Tha Byu 2015 DRD

Road Kanpetlet Kyi Taw Khar Pan 2015 DRD

Road Pinlebu Bunt Bwe Kon Yae Yun Pyit 2015 DRD

Road Ann Sa khan maw Sa khan maw 2015 DRD

Road Ann tat taung tat taung 2015 DRD

Road Htantabin Yae Paw Taung Kan Nar Su 2015 DRD

Road Laymyethnar Paein Inn Paein Inn 2015 DRD

Road Tatkone Tha Bye Kone Tha Bye Kone 2015 DRD

Elec Kan pet let Baung bin Baung bin 2014 DRD

Elec Sidoktayar Pha Aing Ma Gyi Su 2015 DRD

Following are the questions to be answered from the Terms of Reference, scope and subsequent instructions, with discussion and analysis presented for each item as appropriate.

8.1 How does the budget and unit costs compare between the NCDDP sub-projects and comparable infrastructure built by other parties? Breakdown the NCDDP sub-projects into comparable groups of similar technical quality and utility.

8.1.1 Building Sub-Projects

Building cost data was gathered at 64 NCDDP SP and 6 CSP sites. A large number of the SPs evaluated are rehabilitation works (43 SP). One of the CSP buildings viewed was a rehabilitation project. The rehabilitation works varied considerably from one another, so that useful unit costs cannot be calculated for these SPs.

Using only new constructions, spreadsheets of aggregated cost effectiveness data were developed and detailed analysis performed. Building square footage costs were calculated based on data usually drawn from village SP files. Some information was gathered at other project offices, mainly NCDDP Township offices. Costs include community contributions.

All NCDDP SP construction costs cited in this section are including local cash or equivalent for in-kind contributions from villagers. If one were to subtract these voluntary contributions from the SP budgets, the infrastructure will become more cost effective when compared to similar constructions by other agencies.

NCDDP new building construction costs lie mostly between 6,700 and 16,700 Kyat/sq.ft. (for 17 NCDDP SP where this data was obtained). There are four outliers at 3,000 and three greater than 25,000 Kyat/sq.ft. The outliers were discounted which produced an average of 10,200 Kyat/sq.ft.

CSP building construction costs lie between 6,950 and 20,000 Kyat/sq.ft., with one outlier discounted at 34,500 Kyat/sq.ft. The average of CSP buildings (5 new constructions evaluated) is 11,950 Kyat/sq.ft.

An examination of photographs taken by the TE teams indicates that the overall quality of the ‘comparable’ buildings constructed by other agencies and funding sources (DRD; Regional Budget; Hluttaw Fund; and Ministry of Education) may be slightly higher than is typical for NCDDP SPs (no explanatory notes were provided by the evaluation team members). This may explain the 17% higher cost for CSP construction than for NCDDP’s efforts.

It is generally understood that the distance of a construction site from a Township is a strong determinant of the unit cost of buildings. The following table compares the evaluated SPs by their remoteness.

Table 8.1.1.1 – Unit Cost for Building by Remoteness (Kyat/sq. ft.)

Not Remote Remote Very Remote Extremely Remote

NCDDP 8,600 (3 SP) – Note 1 10,900 (6 SP) 11,700 (6 SP) 7,500 (2 SP)

Comparable 34,500 (1 CSP) 13,600 (2 CSP) 7,000 (1 CSP)

Note 1: two SPs were removed from the averaging as they were very much higher than the others.

Discussion:

The NCDDP SPs that were evaluated provide some evidence that the hypothesis of increasing costs with increasing remoteness holds true, with the exception of two schools in Extremely Remote villages. A review of the photographs from these extremely remote NCDDP sites (both in Kanpetlet) show two well-built wooden school structures, but nothing visibly cheaper than as seen in photos from SPs in the other degrees of remoteness. There were no notes made on the SP data input sheets to indicate any special conditions that might have made the two Extremely Remote schools less expensive than their peers.

The two NCDDP SPs discounted in the averaging for Not Remote villages were a small toilet block in Pinlebu and a Kyunsu medical clinic that was constructed with reinforced concrete columns and beams and a steel trussed roof, resulting in much higher unit costs.

The comparable SPs do not provide much valuable data upon which judgments can be made. There were no photographs submitted of the remote CSP with large unit cost; this SP can likely be viewed as an anomaly and outlier. The two Very Remote CSPs have an average unit cost that is slightly above the NCDDP cost.

It would appear from this analysis that NCDDP costs for building construction are consistent with those of agencies and therefore are cost effective.

Bridge

There is a large variety in the size and material used for the construction of the bridges that were evaluated in this study. It is necessary to compare bridges that are of generally equal size and constructed of similar materials.

Two of the three CSPs evaluated are not comparable to NCDDP bridge SPs, in that they use large steel beams to support the decks (one of the decks is wood, the other is concrete). All of the NCDDP bridges use either reinforced concrete (7) or wooden beams (3). Only one of the CSP bridges was constructed using reinforced concrete.

Three NCDDP SPs of this type are ocean jetties. There were no comparable jetties evaluated.

The unit costs for NCDDP reinforced concrete bridge construction lie between 20,800 and 38,500 Kyat/sq.ft with an average of 27,600 Kyat/sq.ft.

The single comparable reinforced concrete bridge SP (by DRD) has a unit cost of approximately 41,700 Kyat/sq.ft. No photographs were submitted from the TE team to verify the quality or condition of this CSP relative to NCDDP SPs.

Although the SP sampling is small with few comparisons possible, it would appear from this analysis that NCDDP costs for bridge construction are in line with those of DRD and therefore are cost effective.

Water Supply

Water supply systems are constructed in a variety of ways, with different methodologies being used to access clean water, move it to a village, and deliver it to a user group.

The primary methodology used in NCDDP SPs are gravity-fed systems, where water is gathered at a mountain source and piped to a village. This study examined 20 NCDDP gravity-fed water systems and a single CSP of this type.

The second most common water system evaluated in this study are mechanically drilled boreholes. There were 12 borehole NCDDP SPs evaluated, as well as 2 CSPs.

NCDDP construction costs for gravity-fed water systems (GFWS) vary greatly according to the specific infrastructure required for each scheme. Also complicating the analysis is the fact that the data gathered by the TE teams was not complete for all aspects of every system. This is understandable and is due to many factors: incomplete SP paperwork, poor plans, non-existent as-built record drawings, buried infrastructure, insufficient time at SP sites for checking and confirmation, etc.

NCDDP gravity-fed systems were sorted according to the length of the transmission/distribution pipe from the source and within the village. This produced a spread of costs that makes sense, although it must be noted that the range of SP budget within each of the pipe lengths is sometimes wide.

Table 8.1.1.2 – Gravity-Fed Water System Unit Cost/Household by Pipe Length

Length of Pipe 0 – 1,000 ft. 1,000 – 2,000 ft. 2,000 – 5,000 ft. > 5,000 feet

Kyat/Household 13,289 15,181 32,861 94,774

Number of SP 6 4 4 2

Note: four SP obvious outliers were discounted to produce these averages.

Discussion:

The spread of average costs in the table above is a logical progression of increasing costs with lengthier transmission and distribution systems. The discounting of four SP to produce these figures should not be forgotten however, nor the High/Low fluctuations within each of these pipe length divisions. For example, in the 0 – 1,000 ft. range, the High/Low was 22,779/3,564 Kyat/HH; for 1,000 – 2,000 ft., it was 25,275/8,276. The ranges for 2,000 – 5,000 ft. and > 5,000 ft. were more reasonable.

GFWS can also be assessed on a cost/tapstand basis. Thirteen (13) SP evaluations provided sufficient data to make this calculation. Two high outliers were discounted, to produce an average of 32,350 Kyat/tapstand for gravity-fed water supply NCDDP sub-projects. The outliers were 88,500 and 101,000 Kyat/tapstand for two SPs in Kanpetlet (the reasons for their larger unit costs are not clear from either evaluation commentary or photographs). The high/low costs are 61,224 and 8,276 so, again, this average must be carefully considered when using this average to assess new SP proposals. The single GFWS CSP evaluated did not feature the construction of any tapstands; the SP involved only the installation of a lengthy transmission pipe from a source to a village (no tapstand information was reported). There were no truly comparable GFWS SP by other agencies evaluated during this study and thus the study cannot make a judgment about the cost effectiveness of NCDDP GFWS investments.

Unit costs for borehole SPs are quite variable. Costs can be quite low if suitable groundwater resources are found near surface, or quite high if multiple holes need to be drilled to find sufficient quantities or depths to groundwater are great. Total costs can also be affected by the number of completed wells desired (as many as 5 in one of the evaluated villages). Nine NCDDP borehole SPs were evaluated in this study, plus a single borehole CSP in Pinlebu.

The unit cost for NCDDP borehole SPs in regards to number of households serviced are approximately 28,000 Kyat/HH, without discounting any outliers (two SPs were approximately 48,500 Kyat/HH and two were approximately 11,700 Kyat/HH). A similar cost for the single borehole CSP evaluated is 34,000 Kyat/HH so it is apparent that the NCDDP borehole programme is cost effective.

Other types of water systems include dug wells equipped with hand pumps (one SP in Pinlebu), diesel pumps from surface water sources (Mon River and two (or three? – incomplete notes make this unclear) systems utilizing existing dug wells in the middle of rice fields), and two ponds in Htantabin where villagers access water from shallow ponds.

It is evident that each water system is quite unique. While some generalizations can be made and conclusions drawn, experienced engineers and technical personnel need to examine each system’s plans and specifications together with the budget in order to determine the true cost effectiveness of SP proposals.

Road

There were 72 NCDDP and 7 CSP roads evaluated. The CSP roads were all constructed by DRD. The type of materials and construction methodologies differed from SP to SP. Following is a table outlining the variety of roads evaluated.

Table 8.1.1.3 – Road Construction Materials (Number of SP Evaluated)

Earth Gravel Macadam Concrete

NCDDP 10 17 4 41

Comparable (DRD) 2 1 4

NCDDP earth road construction costs lie between 34 and 84 Kyat/sq. ft. The average for NCDDP earth roads is 60 Kyat/sq. ft.

NCDDP gravel road construction costs lie between 231 and 675 Kyat/sq. ft. (discounting one outlier at 1,324). The average for NCDDP gravel roads is 394 Kyat/sq. ft.

NCDDP macadam road construction costs lie between 573 and 696 Kyat/sq. ft. The average for NCDDP macadam roads is 649 Kyat/sq. ft.

NCDDP concrete road construction costs lie between 667 and 3,555 Kyat/sq. ft. (discounting 3 outliers at 157, 164 and 5,020 Kyat/sq. ft.). The average for NCDDP concrete roads is 1,320 Kyat/sq. ft.

Similar unit costs were calculated for the CSP roads that were evaluated. The following table presents these figures for comparison.

Table 8.1.1.4 – Average Unit Costs for Different Road Building Materials (Kyat/sq.ft.)

Earth Gravel Macadam Concrete

NCDDP 60 400 650 1,320

Comparable (DRD) 170 950 1,255

Discussion:

Comparing the NCDDP road SPs with CSPs by construction material, one can see an approximate 2.5 scale factor when contrasting NCDDP earth and gravel road works with that of DRD, while concrete roads are almost equal. The reasons for the large discrepancy for earth and gravel roads may be discerned by viewing the photographs from the few DRD roads (two earth and one gravel). It is evident that the DRD roads have been constructed using heavy machinery, whereas the NCDDP roads have been largely labour-based. These “comparable” roads have not been constructed using similar methodologies and thus are not actually comparable. The end users of the three earth/gravel DRD roads would appear to be larger vehicles than are generally planned for NCDDP road SPs (the two DRD earth roads are 18 ft. wide and the gravel 12 ft., whereas most NCDDP road widths are between 6 and 12 ft.).

Bearing in mind the foregoing comparison of the final end products of the two road delivery models, it does appear as though the NCDDP earth and gravel road building program is cost effective for its user group.

The NCDDP concrete road building program has also been shown to be about 5% less cost effective than that of DRD’s normal road delivery model.

Electricity

There were 23 NCDDP electricity SPs and 3 CSP evaluated. The CSP were all sponsored by DRD. The electrical SPs were in a number of different types, as shown in the following table.

Table 8.1.1.5 – Electrical Sub-Project Types (Number of SP/CSP)

Grid Extension Genset Solar Panel Mini-Hydro

NCDDP 6 8 6 3

Comparable (DRD) 3

Some SPs simply extend electrical lines from the existing government electrical grid. NCDDP grid extension costs lie between 40,000 and 70,000 Kyat/HH. The average for a NCDDP grid extension is 53,000 Kyat/HH.

NCDDP genset installation and electrical distribution costs lie between 57,200 and 125,500 Kyat/Household (HH). The average for a NCDDP genset SP is 85,000 Kyat/HH.

NCDDP solar panel installation and electrical distribution costs lie between 99,000 and 285,000 Kyat/HH. The average for a NCDDP solar panel SP is 156,000 Kyat/HH.

NCDDP mini-hydro generator construction and electrical distribution costs lie between 108,000 and 313,000 Kyat/HH. The average for a NCDDP mini-hydro SP is 212,000 Kyat/HH.

The CSPs evaluated were all solar panel installation schemes. Three CSPs were examined and their costs lie between 80,000 and 200,000 Kyat/HH. The average solar panel CSP is 155,000 Kyat/HH. These costs have been summarized in the following table.

Table 8.1.1.6 – Average Unit Costs for Different Electrification Methods (Kyat/HH)

Grid Extension Genset Solar Panel Mini-Hydro

NCDDP 53,000 85,000 156,000 212,000

Comparable (DRD) 155,000

Discussion:

Where the national electricity grid is nearby, it can be readily seen that an extension scheme is about 40% cheaper than the next option, a genset (along with less ongoing costs also). Solar panels are slightly less than twice as expensive as a genset, although lesser ongoing costs over the long term may allow this option to become cost effective with gensets over time (ongoing costs are not examined in this evaluation). Mini-hydro installations were found to be the most expensive option, although the three SPs of this kind had very different costs (high/low of 313,000/108,000 Kyat/HH).

Based on the findings above, it can be seen that NCDDP solar electrification SPs are cost effective when compared to similar schemes by a comparable agency. The remaining NCDDP electrification SP types could not be compared with others.

8.2 Are investments implemented through community force account (CFA) more competitive than those implemented by contractors, when the cost of investments, capacity development and supervision, and the cost and quality of O&M, are taken into account?

The majority of SPs evaluated during this study were implemented through CFA rather than by contractors. The following table provides a breakdown of the SPs by implementation modality.

Table 8.2.1 – Evaluated SP Types by Implementation Modality (Number of SPs)

Building Bridge Water Supply Road

Electricity Total

CFA 60 13 35 65 17 190

Contractor 4 2 1 7 6 20

The SP types were sorted by these implementation modalities vs. the unit costs for each SP type, to determine the cost competitiveness of NCDDP construction methodologies. The unit costs are derived from the budget information contained in the village SP files, which includes the costs for capacity development and supervision. No village files contained O&M costs, so these are not included in the following figures (for both construction modalities).

Table 8.2.2 – Unit Cost of SP Types by Implementation Modality

(Number of SP) Building

Kyat/sq. ft. Bridge

Kyat/sq. ft. Water Supply (Gravity Fed) Kyat/HH Water Supply (Borehole)

Kyat/HH

CFA 7,250 (48) 26,700 (6) 30,300 (23) 26,000 (12)

Contractor 7,070 (3) 32,800 (1) 51,724 (1)

(Number of SP) Road

(Earth)

Kyat/sq. ft. Road

(Gravel)

Kyat/sq. ft. Road

(Macadam) Kyat/sq. ft. Road

(Concrete) Kyat/sq. ft.

CFA 77 (4) 404 (10) 649 (4) 1,322 (37)

Contractor 43 (4) 290 (1) 1,265 (1)

Table continued overleaf.

(Number of SP) Electricity

(Grid Extension)

Kyat/HH Electricity

(Genset)

Kyat/HH Electricity

(Solar Panel)

Kyat/HH Electricity

(Mini-Hydro)

Kyat/HH

CFA 58,400 (4) 85,000 (8) 97,000 (3) 210,000 (2)

Contractor 41,800 (2) 215,000 (3) 214,600 (1)

Discussion:

Building

A small sample of building SPs implemented by contractors appears to be slightly more cost competitive than a large sampling of CFA implemented structures. The difference is less than 2.5%, however. An examination of the photographs from those SPs implemented by contractors does not reveal any major variables that might have influenced this finding.

Bridge

A single bridge constructed by a contractor is approximately 25% more expensive than the average derived from a sample of six CFA SPs. It must be noted that this average is based upon a high/low spread of 38,500/20,800 Kyat/sq. ft., which more than encompasses the contractor’s unit cost.

Water Supply

A single water supply borehole SP (in Tatkon) was handled by a contractor and yielded a unit cost almost double that of the average of 12 SPs that were implemented through the CFA construction implementation modality (51,700 vs. 26,000 Kyat/HH). Field notes indicate that the Tatkon drill rig encountered many dry holes before finally finding adequate groundwater resources, so that use of this data is uncertain.

The evaluation sampling contained no gravity fed water supply (GFWS) systems constructed using the services of contractors.

Road

It appears that the contractor-implementation modality produces more competitive road works across all types of road evaluated. Roads constructed of earth and gravel are approximately 56% and 71% cheaper than similar CFA works. A single concrete road was implemented using a contractor and was 5% cheaper than the average of 37 CFA SPs, although the high/low unit costs for CFA is 3,555/667 Kyat/sq. ft., which amply surrounds the contractor’s unit cost. There were no macadam roads implemented by contractors for comparison.

Electricity

The unit cost for a grid extension SP is about 30% lower when using a contractor. This may be a logical situation since the work of extending the high voltage electrical grid is a detailed and exacting process where previous experience will allow cost savings for a contractor. It is likely that the CFA process in some villages encountered unforeseen difficulties during sub-contracting that will have increased costs.

It is unclear why the CFA implementation modality in solar panel installation is so much less expensive than the use of contractors, while the costs for the implementation methods is roughly similar with mini-hydro SPs.

In summary, the CFA implementation modality will be more cost competitive for bridge and solar panel SPs, but less so for roads and grid extension electrical SPs where contractors seem to offer a cheaper option to communities. The two construction implementation modalities seem to be equal when considering the construction of buildings and mini-hydro SPs. No conclusions are possible for water systems.

8.3 Are there community contributions, and if yes, how much, how was it calculated, what forms did these contributions take and what percent of total costs?

Local community contributions to NCDDP SPs are recorded in the village sub-project implementation files. Local contributions were made through cash contributions and the provision of labour and materials. The average community contribution to a NCDDP SP is 7% of the infrastructure’s total budget. Following is a table that shows the percentage of community contribution for each Township.

Table 8.3.1 – Average Community Contribution by Township (% of SP Budget)

Community Contribution

Kanpetlet 7%

Pinlebu 10%

Kyunsu 13%

Sidoktaya 4%

Ann 2%

Htantabin 4%

Namhsan 14%

Laymyetnar 4%

Tatkon 1%

Average (210 NCDDP SPs) 7%

Details of the labour and material contributions can be found in NCDDP forms and in attachments that outline the name of the labourer, the nature of the labour, the type and quantity of materials supplied, and the date of such work. Labour contributions are often for excavation; supply of sand, gravel or stone; general construction activities; etc. Numbers of manpower hours on specific days are recorded, along with calculations of daily contribution amounts based on hourly wages and typical market values for materials.

8.4 Where community contributions are identified in the sub-project documents,

8.4.1 Did the contributions actually occur and were they accounted for properly?

TE team members studied the community contribution documentation to see if there was evidence of NCDDP personnel checking and signing-off on the information. The Operations Manual outlines how voluntary community contributions can be collected to fund SPs. Form F8 is to be completed to record the name of contributor, description of contribution, date, and final amount contributed.

The following table provides data in regards to community contributions. Each line represents data from all SP evaluated in each Township. The final line sums up the full SP sampling for the country.

Table 8.4.1.1 – Community Contributions (CC) and Verification of Accounting

Average SP Budget (Kyat) Average Worth of Community Contribution (Kyat) Community Contrib.

(% of Budget) Verification of CC Accounting

(Evidence in file

of NCDDP checks) (% of SPs)

Kanpetlet 7,989,000 566,607 7% 88%

Pinlebu 4,086,000 406,200 10% 90%

Kyunsu 8,187,000 1,023,832 13% 87%

Sidoktaya 6,167,000 251,762 4% 100%

Ann 5,500,000 109,886 2% 38%

Htantabin 7,484,000 297,053 4% 100%

Namhsan 5,729,000 774,406 14% 90%

Laymyetnar 4,028,000 173,759 4% 80%

Tatkon 10,414,000 101,685 1% 100%

All Townships 6,592,000 434,859 7%

Discussion:

It can be seen that the amounts of voluntary contributions to NCDDP SP budgets are highly variable according to Township, from a low of 1% and 2% in Tatkon and Ann respectively, to a high of 13% and 14% in Kyunsu and Namhsan respectively. No commentary was recorded on the technical inspection forms to suggest why these differences are so.

The filing of proper documentation to record and verify these contributions is, for the most part, adequately performed in Townships, with the notable exception of Ann where only 38% of SP files were found to contain the proper documents with respect to voluntary contributions. Other Townships can also improve their record-keeping in this respect to achieve 100% compliance with the OM.

Recommendation 9: NCDDP field staff training should emphasize the importance of village SP implementation file review during monitoring visits. Community contributions should be checked and signed-off on a regular basis.

8.4.2 Are the costs of the contributions as reported reasonable for the community inputs?

As noted above, recipient communities have voluntarily contributed cash and labour or materials toward the SPs being constructed in their villages, contributing an average of 7% of the budgets. This can also be analyzed by looking at the number of communities where such contributions took place. The following table provides the actual number of villages where either cash or labour/material contributions took place, along with a percentage for that Township.

Table 8.4.2.1 – Community Contributions (Number and % of SP villages)

(Number of SPs) Cash Contribution Labour and Materials Contribution Materials Transport

Contribution

Kanpetlet (25) 2 8% 4 16%

Pinlebu (28) 1 4% 10 36%

Kyunsu (30) 7 23% 9 30% 2 7%

Sidoktaya (19) 3 16% 10 53%

Ann (16) 2 13% 7 44%

Htantabin (22) 3 14% 6 27%

Namhsan (19) 5 26% 5 26% 1 3%

Laymyetnar (29) 6 21% 6 21%

Tatkon (22) 2 9% 1 5%

Discussion:

Cash contributions vary from a low of 4% of SP budget in Pinlebu to a high of 26% in Namhsan. Labour and materials supply is a low 5% of SP budget in Tatkon to a high of 53% in Sidoktaya.

In order to determine if these contributions are reasonable it could be argued that a study of each recipient village’s or Village Tract’s resources should be conducted. The study might count the number of motorcycles, trucks or other indicators of financial worth, and thus predict the availability of villagers to make inputs of labour or local materials. Those village populations more able to make community contributions might be readily apparent through such a survey. Repeating the survey a year or two after construction of road, irrigation or other economic generation SPs would also be useful in these regards.

Without this background social data the only parameter that can be assessed is the size of the community contributions as compared to the requirements of the NCDDP Operations Manual. Based on the evaluation sample’s community contribution rate of 7% of total SP budget, we conclude that the size of the contributions is reasonable for the size of the NCDDP investments.

8.4.3 Were there additional community contributions not reported; and

8.4.4 Assess whether contractors were ever paid for the part of works carried out with community contributions.

Village committee members were questioned during interviews in regards to any other village contributions to the SP construction that may not have been reported. Committee members deferred to the official records, indicating that there were no other contributions of which they were aware. Committee members also confirmed at all SPs evaluated that contractors were not paid for any of the work covered by the voluntary community contributions.

8.5 Were community contributions an important factor in determining the cost effectiveness of NCDDP sub-projects relative to similar sub-projects supported by others?

The value of the village’s voluntary contributions, usually labour or supply of local construction materials, is counted as part of the NCDDP SP budget. The inclusion of these sums, sometimes cash contributions but usually the monetary equivalent of labour or materials provided by villagers, acts to lessen the cost of the infrastructure to the NCDDP.

Lines 1 in Table 8.5.1, below, provide the NCDDP unit costs as calculated in Section 8.1, above, for various salient types of infrastructure. The second line calculates the actual NCDDP investment by subtracting the average local contributions (7%) and then calculating a new unit cost based on floor area, number of households, etc. The effect of this subtraction is to lower the average unit cost of NCDDP infrastructure. The table compares the result with those unit costs of other agencies (lines 3). Several sub-types of infrastructure do not have comparisons (macadam road, gravity-fed water supply, etc.)

Table 8.5.1 – Community Contributions’ (CC) Effect on Cost Effectiveness

(Number of SP) Building

Kyat/sq. ft. Bridge (Reinforced Conc)

Kyat/sq. ft. Water Supply (Borehole)

Kyat/HH

1 NCDDP unit cost 10,200 (15) 26,700 (6) 28,000 (12)

2 NCDDP unit cost less CC 9,500 24,800 26,000

3 Comparable 11,950 (5) 41,700 (1) 34,000 (1)

Road

(Earth)

Kyat/sq. ft. Road

(Gravel)

Kyat/sq. ft. Road

(Concrete) Kyat/sq. ft.

1 NCDDP unit cost 60 (8) 400 (11) 1,320 (38)

2 NCDDP unit cost less CC 56 370 1,250

3 Comparable 170 (2) 950 (1) 1,255 (3)

Electricity

(Solar Panel)

Kyat/HH

1 NCDDP unit cost 156,000 (6)

2 NCDDP unit cost less CC 145,000

3 Comparable 155,000 (3)

Discussion:

It can be seen that these community contributions enhance the cost effectiveness of NCDDP buildings, reinforced concrete bridges, boreholes, roads and electricity schemes. In the case of concrete road infrastructure, the community contributions have moved the NCDDP works from a slightly-more expensive position to a slightly-less vantage point.

The cost effectiveness of all NCDDP infrastructure types is improved by including the effect of voluntary contributions. NCDDP concrete roads are more cost effective than DRD roads after deduction of the community contributions.

8.6 How reasonable are the costs for materials, transport, labor and other inputs?

Upon reviewing the costs derived in 8.1 and the analysis in 8.5, above, it is evident that the NCDDP costs for materials, transport, labour and other inputs are very much in line with those of comparable agencies. Considering the relative youth of NCDDP’s design and construction program, it can be argued that the unit-cost competitiveness of the majority of their works to date is highly admirable and wholly reasonable.

8.7 Were SPs designed to maximize community benefits through employment of local labor, procurement of local materials, or other means?

The designs for NCDDP SPs come from a number of sources, but all are based upon local construction practices which frequently are labour-based methodologies. For example, most excavations are performed by local labourers rather than using machinery; where concrete is used, it is mixed by hand or small machine and placed in formwork using buckets. These techniques enable and promote the use of local unskilled labour. Construction materials are also sourced locally whenever possible, including sand, rock and wood, providing additional benefits to the community. Some of this work by villagers is provided as a community contribution to the SP financing, but much of it is paid as skilled or unskilled labour working for a contractor. The social benefits through employment of local labor and procurement of local materials contribute to the effectiveness of NCDDP SPs in maximizing community benefits.

It is apparent that NCDDP SP are designed to utilize as much local labour and locally sourced construction materials as possible. This construction modality increases the local sense of ownership of the infrastructure which, in turn, benefits the ongoing operation and maintenance of the facilities.

8.8 Were SP designs and specifications selected to maximize value for money? Would other designs, technologies or methods have provided greater value?

The findings of this cost effectiveness study show that the NCDDP model of community SP implementation produces rural infrastructure of a generally suitable technical quality for costs that are reasonable when compared to those of other agencies. Earth and gravel roads demonstrate this most readily, as do borehole water systems, buildings and solar panel schemes when the average community contributions are considered.

It is evident from cost comparisons with comparable infrastructure by others that NCDDP SPs have been designed, specified and constructed to maximize value for money. The majority of the designs, technologies and construction methods are suitable for the NCDDP’s clientele, with several changes possible that might offer greater value. The changes recommended to the NCDDP within this report will provide more sustainable infrastructure to villages and thus provide greater long-term value and cost effectiveness.

The social benefits through employment of local labor and procurement of local materials contribute to the cost effectiveness of the SPs. The CFA construction modality increases the local sense of ownership of the infrastructure which, in turn, benefits the ongoing operation and maintenance of the facilities.

9 Compliance with Environmental and Social Safeguards

Following are the questions to be answered from the Terms of Reference and scope, with discussion and analysis presented for each item as appropriate.

9.1 Proper documentation and recording of Environmental Code of Practice (ECoP) and the Safeguard Checklist, and the verification and monitoring by the Township NCDDP office of contractor/community compliance with ECoP.

Field Tool 3 provides a questionnaire where the quality of the infrastructure, its site selection, and the process under which the construction took place can be assessed in regards to environmental and social considerations. The TE team referenced the Environmental and Social Management Framework (ESMF), Annex 8 Environmental Codes of Practice (ECoP).

Project files were examined for proper documentation, and evidence of monitoring and verification by Township NCDDP officials of community and contractor compliance with the ECoP. Table 9.1.1, below, presents a summary of these findings for an aggregate of all NCDDP SP evaluated in each Township.

Table 9.1.1 – Summary of Environmental and Social Safeguards Findings by Township (% of Sub-Projects)

(Number of SPs) ECoP contained in SP file, with evidence of monitoring Environmental Management Plan included in the file (Form PC 15) and followed onsite Safeguard Screening Form PC 13 in sub-project file

Kanpetlet (25) 100% 0% 100%

Pinlebu (28) 96% 4% 96%

Kyunsu (30) 76% 21% 93%

Sidoktaya (19) 100% 0% 100%

Ann (16) 100% 0% 100%

Htantabin (22) 100% 0% 100%

Namhsan (19) 100% 0% 100%

Laymyetnar (29) 100% 0% 100%

Tatkon (22) 100% 0% 100%

Total (210) 96% 3% 99%

Discussion:

The TE teams’ assessment of NCDDP SP files in the villages shows that, for the most part, SP implementation teams are maintaining properly organized documentation – EcoP and Safeguard forms have been completed in 96% and 99% of SP sites, respectively.

The inclusion of an Environmental Management Plan is stipulated in the OM if the completion of the Safeguard Screening Form (Form PC 13) records any ‘Yes’ answers to a series of environmental questions. As can be seen in the final column of Table 9.1.1, almost all SP files were found to contain PC 13. The fact that very few SP files contain an EMP (only 3% of SP files contain this document) indicates that almost all SPs have been considered very low risk within the Safeguard Screening form.

9.2 Loss of land or private assets, the scale of impact, whether or not they are addressed through voluntary donations and if so, whether all conditions of voluntary donations as provided in the operations manual are met.

Field Tool 3 features a section where evaluators confirm the land status of the SP and whether or not transfers of land have been done according to NCDDP policy. There are three choices for this query: voluntary land donation conditions met (Yes), voluntary land donation conditions not met (No), or no land donation required (Not Applicable). NCDDP policy has been followed at all sites visited. A majority of SPs (83%) are constructed on existing public lands so that no land donations are required (see Table 9.2.2 for a breakdown of SP types requiring land donations).

Table 9.2.1 – Summary of Land Donations and Documentation (% of SP)

(Number of SPs) Voluntary land donation conditions met. Form PC 14 in file No land donation necessary for sub-project

Kanpetlet (25) 0% 100%

Pinlebu (28) 4% 96%

Kyunsu (30) 14% 86%

Sidoktaya (19) 21% 79%

Ann (16) 31% 69%

Htantabin (22) 0% 100%

Namhsan (19) 11% 89%

Laymyetnar (29) 45% 55%

Tatkon (22) 27% 73%

Total (210) 17% 83%

Table 9.2.2 – Land Donated by SP Type

Building Bridge Water Supply Road Electricity

Land Donation Required (Number of SPs) 4 1 22 2 6

Total SPs Evaluated 64 15 36 72 23

Discussion:

Townships seem to vary greatly in the necessity for land donations to take place. Sub-projects in Kanpetlet and Htantabin were all found to have been constructed completely on existing public lands (according to SP documentation), whereas Laymyetnar and Ann required many pieces of land to be donated (45% and 31% of their SPs respectively). It can be noted from Table 9.2.2 that the predominant types of SP requiring land donation are water supply (61% of water supply SPs require land donation) and electrical schemes (26%). Kanpetlet, Htantabin and Pinlebu (the Townships with the lowest requirements for land donations) share a similarity of hosting relatively fewer water and electrical SPs than do their peers (thus offering some confirmation of the low land donation rates in these Townships, as reported in Table 9.2.1).

The TE teams were not instructed to question the veracity of the land donation information provided by VPSC representatives, and it is likely that few individual affected landowners were quizzed for confirmation of the voluntary nature of land transfers. No reports of disgruntled or unhappy landowners were recorded by the evaluation teams.

9.3 Verification of whether any adverse environmental impacts occurred at the sub-project site, and how they were mitigated.

A thorough examination of the SP and surrounding site was performed as part of the Field Tool 1 investigation. Environmental impacts of the SP were observed at this time, along with mitigation measures that were part of the construction. TE team members also reviewed the terms and conditions as set out in ECoP, where it was available, to verify the SP’s environmental requirements and mandated mitigation measures.

Table 9.3.1 – ECoP Confirmation

(Number of SPs) Site inspection confirms that ECoP followed during construction

Kanpetlet (25) 100%

Pinlebu (28) 96%

Kyunsu (30) 76%

Sidoktaya (19) 100%

Ann (16) 100%

Htantabin (22) 100%

Namhsan (19) 100%

Laymyetnar (29) 100%

Tatkon (22) 100%

Total (210) 96%

Discussion:

The majority of SPs evaluated, 96%, were found to have adequately addressed issues that were raised in the ECoP for each individual site. One (1) SP in Pinlebu and 7 in Kyunsu were found to be lacking in this environmental documentation and/or verification and monitoring at village sites. Facilitators in Pinlebu and Kyunsu may benefit from refresher training courses in regards to the importance of the various environmental and social frameworks that govern the work of the NCDDP.

When evaluators noticed specific mitigation measures being taken, notes were made on the field tools. These notes were later transcribed to digital submissions. All field notes of this sort are contained in Annex 5, Commentary From SP.

Recommendation 10: All NCDDP environmental and social safeguard checklists and forms must be completed for each SP site. Environmental monitoring activities should be ongoing during the SP construction, with notes to file as appropriate. Refresher training courses should emphasize the importance of this documentation.

 

10 Operation and Maintenance/Sustainability

Year 1 sub-projects in Kanpetlet, Kyunsu and Namhsan were evaluated with special regard to the O&M/sustainability aspects of the infrastructure.

Following are the questions to be answered from the Terms of Reference and scope, with discussion and analysis presented for each item as appropriate.

10.1 Is the current condition of SP infrastructure good, fair or poor?

The current condition of the Year 1 SP infrastructure in Kanpetlet, Kyunsu and Namhsan was judged in regards to O&M and sustainability using Field Tool 1, during the technical assessment as discussed in Section 7.1, above. The quality of the O&M was assessed against the specifications and documentation contained in the SP proposal and files, using Meets Spec, Slightly Below Spec, and Below Spec (analogous to Good, Fair and Poor). The following table shows the data gathered for this item.

Table 10.1.1 – O&M Quality Rating

Kanpetlet

(14 SP) Kyunsu (10 SP) Namhsan (6 SP)

Meets Specification (Good) 12 9 5

Slightly Below (Fair) 2 1 1

Below Specification (Poor) 0 0 0

Discussion:

The Year 1 SPs were generally judged to be meeting the requirements as set out in the O&M Committee documents. Several SPs were evaluated as not quite meeting the terms and conditions of this documentation. Written commentary for two of these electrical SPs (of four Year 1 electrical SPs) indicates that invertors have been “destroyed” and not replaced. Comments were not provided by the evaluation teams to explain their rationale for the other two of these lower ratings (a building and a road). An examination of photographs submitted from these sites also did not shed light on these ratings. A summary of the specific SP types rated for O&M by Township are in the following tables.

Table 10.1.2 – O&M Quality Rating by Township and SP Type (Number of SP)

Kanpetlet Meets Specification (Good) Slightly Below (Fair) Below Specification (Poor)

Building 4

Water Supply 3

Road 5

Electricity 2

Kyunsu Meets Specification (Good) Slightly Below (Fair) Below Specification (Poor)

Building 2 1

Bridge 2

Water Supply 1

Road 3

Electricity 1

Namhsan Meets Specification (Good) Slightly Below (Fair) Below Specification (Poor)

Building 2

Road 2 1

Electricity 1

Notwithstanding the above, it is evident that the current condition of the majority of NCDDP infrastructure is good.

10.2 Have any major repair or restorative maintenance/ rehabilitation works conducted since the completion of civil works or does the current condition require such works? If so, what are the causes of defects? Break down the causes of defects into environmental/ natural factors; technical defects in design, implementation or materials.

Year 1 SP village O&M Committee members were questioned by the TE team in regards to major repair work that had been performed or that was considered necessary. Major repairs are those items requiring attention beyond routine maintenance. Major repairs normally involve expenditures of cash (whereas routine maintenance tasks are usually labour-based). Where major repairs were necessary, the causes for these circumstances were explored by the evaluators, breaking down the defects into environmental or natural factors; improper design; faulty construction techniques; or poor materials. Following are tables presenting summaries of this information.

 

Table 10.2.1 Major Repairs, by Sub-project Type – Number of SPs affected (of SP evaluated)

Building Bridge Water Supply Road Electricity

1 Major repairs or rehabilitation performed, Year 1 SP 0 (9) 0 (3) 0 (3) 0 (11) 0 (4)

2 Major repairs or rehabilitation required, Year 1 SP 0 (9) 0 (3) 0 (3) 0 (11) 1 (4)

3 Major repairs or rehabilitation performed, Year 2 SP 0 (55) 0 (13) 4 (32) 1 (61) 0 (19)

4 Major repairs or rehabilitation required, Year 2 SP 1 (55) 0 (13) 2 (32) 3 (61) 0 (19)

Table 10.2.2 Major Repair Cause

Environmental Design Construction Materials

5 Building (1 SP) ✓ (1)

6 Water Supply (6) ✓ (1) ✓ (2) ✓ (1)

7 Road (4) ✓ (1)

8 Electricity (1 SP) ✓ (1)

Discussion:

The only SP from the Year 1 sample that requires major repair work is an electrical system in Namhsan (line 2 above). This is indicative of the generally satisfactory quality of the infrastructures constructed during that cycle. Written commentary from the evaluation of that mini-hydro SP indicates that the dynamo generator has broken. An estimate of 200,000 Kyat has been made to fix this equipment. The cause of the failure of the dynamo was marked as pertaining to the materials specified (line 8 above).

Many more major repairs have taken place or are required for the Year 2 SPs evaluated. Table 10.2.1, lines 3 and 4, show that five major repairs have taken place already (4 on water systems (Ann, Tatkon and 2 in Kyunsu) and 1 on a road SP in Tatkon) while another six major repairs have not yet been undertaken (one building (Ann), 2 water supply (Tatkon, Kyunsu) and three road works Laymyetnar, Htantabin, Sidoktaya)). The main causes of these defects appear to be construction-related (4 instances), with faulty materials being blamed in two cases and one an environmental problem. (Complete field data was noticeably lacking for this portion of the survey, for unknown reasons. Little written commentary was provided by the field teams to explain the rationale of the findings.)

Recommendation 11: NCDDP Engineering Department should investigate the cause of the failure of the dynamo in Ho Nam village, Namhsan. If the equipment specified for this application was not suitable for the circumstances, NCDDP should fund a replacement dynamo for this SP.

10.3 If any O&M works have been done, who did what O&M works?

All SP O&M Committees reported that O&M works had been 100% village labour and that materials had been purchased by the village. No contractors had been hired to perform any of the works.

10.4 Was any routine maintenance (wear and tear; replacement of consumables) carried out on the sub-project?

O&M committee members were questioned regarding the routine maintenance activities that had been undertaken in the past. Following is a table that lists individual maintenance tasks for each SP type and for each year of construction, along with some notes for each.

Table 10.4.1 – Routine Maintenance Activities – % of Active O&M Committees

Building Routine O&M Activities

Roof repair Mechanical Plumbing Concrete repair Plaster repair Washing Painting Drainage

Year 1 SP 11% 0% 0% 11% 11% 22% 22% 0%

# of SPs (of 9) 1 0 0 1 1 2 2 0

Year 2 SP 2% 0% 0% 2% 0% 15% 15% 0%

# of SPs (of 55) 1 0 0 1 0 8 8 0

Building routine maintenance notes: It is good to see an increase in the percentage of O&M Committees working on almost all routine tasks as time goes by. Many Year 2 SPs have only recently been finished, so low percentages of active committees may be understandable.

No O&M Committees report any work on mechanical or plumbing – two systems that frequently break down (due to low quality materials having been purchased). It may be that these systems have not yet started to fail.

It is encouraging to see O&M Committees washing and painting so early in a building’s life cycle, with both of these indicators increasing into a SP’s second year (Year 1 SPs are into their second year).

Bridge Routine O&M Activities

Deck repair Concrete repair Drainage Apron and road repair Support structure Railings Erosion protection

Year 1 SP 33% 33% 0% 33% 0% 33% 33%

# of SPs (of 3) 1 1 0 1 0 1 1

Year 2 SP 0% 0% 0% 0% 0% 0% 8%

# of SPs (of 13) 0 0 0 0 0 0 1

Bridge routine maintenance notes: Again, the increased activity of Year 1 O&M Committees over the Year 2 cohort is a good sign of things to come.

The routine maintenance of Erosion Protection measures of Year 2 Committee is encouraging, as is the increase in percentage of Year 1 SP committees. The sample size is small, however, so vigilance on the part of NCDDP monitors is warranted.

Water Supply Routine O&M Activities Reservoir cleaning Pipe repair Pipe flushing Valve exercising Mechanical repair Filter bed replacement Painting Drainage

Year 1 SP 100% 67% 0% 0% 0% 0% 0% 33%

# of SPs (of 3) 3 2 0 0 0 0 0 1

Year 2 SP 53% 22% 0% 3% 9% 6% 0% 3%

# of SPs (of 32) 17 7 0 1 3 2 0 1

Water supply routine maintenance notes: Some of the routine maintenance tasks noted in the field tool do not apply to all water supply systems being evaluated, so that low activity of some items may not be reflective of the needs of each system. There were, for example, no filter beds constructed in Year 1 SPs.

Road Routine O&M Activities

Pot hole/surface repair Erosion control of shoulders Erosion control of slopes Drainage Vegetation removal Signs Minor repair culverts/walls Regrading and re-gravelling

Year 1 SP 27% 27% 0% 27% 36% 0% 0% 0%

# of SPs (of 11) 3 3 0 3 4 0 0 0

Year 2 SP 23% 38% 11% 15% 3% 0% 2% 8%

# of SPs (of 61) 14 23 7 9 2 0 1 5

Road routine maintenance notes: Fairly good percentages of newly completed Year 2 SPs seem to be paying attention to erosion control measures for road shoulders and slopes, as well as pothole/surface repair and drainage. Year 1 road beneficiaries are starting to deal with vegetation, although this percentage is low.

Many NCDDP roads are concrete, so that regrading/re-gravelling is not necessary.

Electricity Routine O&M Activities Vegetation Removal Mechanical Servicing Washing Conductor Repair

Year 1 SP 25% 25% 0% 0%

# of SPs (of 4) 1 1 0 0

Year 2 SP 5% 32% 0% 0%

# of SPs (of 19) 1 6 0 0

Electricity routine maintenance notes: The activities of the Year 1 O&M Committees is low (although the sampling size is small), which should serve as a reminder to NCDDP that further O&M training sessions may be appropriate for this type of infrastructure.

Discussion:

An overview of the brief notes above shows that for the most part O&M Committees are performing routine maintenance activities in an appropriate manner, with increasing activity as time goes by. It does appear as though further O&M training sessions might be useful for some types of infrastructure, particularly water supply and electrical SPs.

Recommendation 12: NCDDP Engineering Department should provide refresher training sessions to O&M Committees on the 1-year anniversary of the completion of a SP, performing a rigorous inspection of the works and then offering pointers as to how regular periodic maintenance can increase the usefulness and functionality of the infrastructure. Engineering inspections of the systems should take place prior to these sessions so that the course material can be adjusted to suit each individual site.

10.5 Is the quality of the O&M Plan sufficient? In particular, does it address both normal wear and tear, routine maintenance and replacement of spare parts, and reactive maintenance/ capital repair? Do the O&M plans adequately cover the requirements over 3 – 5 years of operation, and clearly spell out specific works to be done, and agencies responsible for and expected cost of respective works?

The O&M Plan for each SP was inspected by the TE team and discussed with the O&M committee members present. The Plans were generally found to be filled out and in proper order. Following is a list of the items verified during this examination:

Table 10.5.1 – O&M Plan Adequacy (% of 210 SPs evaluated)

1 Routine maintenance tasks and costs 90% of O&M Plans contained this

2 Major capital repair costs 0.5% contained this (1 SP)

3 Multi-year O&M plan (normally 3 – 5 years) 99% contained this

4 Linkages to appropriate line Ministries 2% contained this

5 Clear division of responsibilities and costs 94% contained this

Discussion:

The high percentage reported for lines 1, 3 and 5 (90%, 99% and 94%) are indicative of the formulaic nature of the documents that comprise the O&M Plan. The completion of these forms is part of a standard NCDDP SP preparation process and much of the information contained within the form is of a typical character and appears to be seldom altered to suit the particular circumstances of an individual SP.

The low percentage associated with the capital repair item (line 2, only 1 SP of 210 had costs noted) is not a poor reflection on the village O&M Committee. Responsibility for this item lies most directly with NCDDP staff who will have aided the village in this document’s preparation. Estimation of major capital repair is a task best suited to senior engineers who will have an understanding of the average life expectancy for the various components of different rural infrastructures. The inclusion of an accurate major capital repair figures, with an explanation of their significance, may prompt O&M Committees to take better care of infrastructure to avoid or delay such expenditures.

Individual O&M Plans do not address details of routine maintenance for each SP type, but rather provide general guidelines. Almost all Plans contain a three-year schedule (line 3 above) but lack specific descriptions of action items and detailed cost estimates.

Very few O&M Plans (4 SPs) contained clear statements that linked line Ministries to the village infrastructure. As seen in line 5, the majority (94%) of Plans were found to give a clear idea that villagers would be responsible for the infrastructure themselves.

10.6 Is there an O&M committee in place and functioning? What are the O&M arrangements? What are the roles and responsibilities (both financial and technical) of local governments/ line agencies and communities? Are roles and responsibilities separated for direct beneficiaries/ users and indirect beneficiaries?

Table 10.6.1 – O&M Committees and Implementation Arrangements

Building Bridge Water Supply Road Elec. All SP

Year 1 O&M Committee in place and functioning 78% 100% 100% 91% 50% 76%

7 of 9 SP 3 SP 3 SP 10 of 11 2 of 4

Year 2 O&M Committee in place and functioning 96% 100% 97% 97% 95% 96%

53 of 55 12 SP 31 of 32 59 of 61 18 of 19

Implementation by:

Villagers 100% 100% 100% 100% 100% 100%

Government forces 0% 0% 0% 0% 0% 0%

Discussion:

The majority of SPs evaluated (94%) reported that the O&M Committee exists and is active. However there is a worrying trend noticed between the Year 1 and Year 2 SPs, where lesser numbers of village SP committee members are reporting that O&M Committees are in place and functioning for building and electrical SPs. The sample size is small, so this may be an anomaly but worth checking while refresher O&M training sessions are being conducted.

All SP committees reported that villagers had provided 100% of the labour and materials for all O&M activities to date. Very few village committees collect indirect beneficiaries fees (only 1 of 210 SPs evaluated) and no commentary was recorded in regards to the roles and responsibilities for this group.

10.7 Was any training provided to communities on O&M (including refresher training), and if so, what types of training were provided? Did communities request and/ or receive technical support from local governments/ line agencies on O&M?

O&M Committee members were questioned about the O&M training that they had received and whether there was any budget allocated toward ongoing training. Villagers reported that the training was generally received shortly after the SP construction, and that refresher sessions have been offered afterwards; no villages had made any requests for extra training. Training consisted of demonstrations of how the infrastructure operates (for water systems, electrical schemes) and routine maintenance activities (cleaning of water reservoirs, solar panel cleaning, clearing of vegetation from water canals, repair of road potholes and shoulders, etc).

The following table presents the data gathered in Field Tool 4.

Table 10.7.1 – O&M Training and Support

(Number of SPs) O&M training received

(% of SP Committees) Ongoing capacity development

(% of SP) Support from line Ministries/Gov’t Agencies

Kanpetlet (25) 100% 100% 0%

Pinlebu (28) 100% 96% 0%

Kyunsu (30) 93% 93% 0%

Sidoktaya (19) 100% 100% 0%

Ann (16) 100% 100% 0%

Htantabin (22) 100% 100% 0%

Namhsan (19) 100% 100% 0%

Laymyetnar (29) 100% 90% 0%

Tatkon (22) 95% 95% 0%

Discussion:

Almost all SP Committees reported that they had received appropriate training and demonstrations of proper operation and maintenance activities (2 SPs in Kyunsu and 1 in Tatkon were identified as lacking these sessions. No villagers reported any involvement or support from government ministries or sector agencies.

10.8 Is an O&M fund in operation? Who holds the funds? What is the current value of these funds? How are contributions made? By whom? Are those expected to contribute able and happy to contribute?

Table 10.8.1 – O&M Costs and Funds in Account

Building

(64 SP) Bridge (16) Water Supply (35) Road

(72) Electricity (23) Average

1 O&M user fee in place (No. of SP

and %) 4 2 16 3 13 18%

6% 13% 46% 4% 57%

2 Current funds in O&M bank account (Average, Kyat) 21,500

(8 villages) 40,300

(4 villages) 62,000

(14 villages) 191,000

(11) 160,000

(11) 130,500

3 Affordability of User Fees

(% that can afford) 97% 100% 100% 100% 100% 100%

4 % of O&M Committee with Funds 13% 25% 40% 15% 48% 23%

Discussion:

O&M user fees are in place in 18% of SP villages (line 1 above), while a slightly larger percentage (23%) hold funds in a bank account (line 4). No explanations were gathered from those villages holding funds without a user fee in place as to where the money came from. It is likely that special collections had been made for specific repairs or maintenance and the funds have not yet been disbursed.

Line 4 provides the percentage of villages that possess O&M funds, by SP type. The final column presents an average for all Townships. As commonly seen, the two ‘utility’ type infrastructure, water supply and electricity, have the highest percentages for both having a user fee system in place and committees with funds available (40% and almost 50% of water system and electrical SPs, respectively, hold funds). Users of these types of infrastructure more readily recognize the value of such systems and are willing to pay to operate and maintain them.

The low values for building and road SP villages (lines 1 and 4) reflect the fact that most people do not feel as ‘connected’ to building and road infrastructure. It is common that villages provide funding for building and road repair works on a more limited and sporadic basis. Most village road maintenance inputs are labour-based and directed at routine tasks, such as vegetation removal and limited pothole repair, etc. Building maintenance is often simply deferred until the roof or mechanical/plumbing fixtures demand attention, prompting special collections from users.

Almost all O&M Committees report (line 3) that all village families can afford the fees. There was a single village in Kanpetlet where the committee estimated that about 10% of their village’s families could not afford to pay a school user fee.

User fees are generally charged once/month or once/year. Those villages that collect such fees were quizzed regarding their schedule of payments. Most villages collected a user fee once/month, an average of about 3000 Kyat/month for water and electrical systems. Road SP O&M Committees did not tend to charge regular fees but instead required labour donated for periodic tasks, such as clearing vegetation and filling potholes with local sources of gravel; etc. Anecdotal information is available on this aspect of user contributions; no specific data was collected by the TE teams on this aspect of user contribution.

10.9 Has the O&M fund been developed based on the consideration of technical requirements?

The SP documentation studied by the TE teams did not make specific references to O&M funding sources nor provide any formula for the calculation of reasonable fees, such as a percentage of construction costs. Villager O&M Committees are allowed to make their own collective decision as to whether or not such a fund should be gathered, the amount of fees, schedule of payments, etc. It does not appear that the NCDDP O&M documentation is directive but rather is a suggested course of action.

Recommendation 13: The NCDDP should consider revising O&M Committee documentation to stipulate activities that must be undertaken according to a routine schedule, with realistic funds allocated for labour and materials.

10.10 Assess whether applicable user fees are affordable to users and sustainable to finance longer term O&M. Did the line ministries contribute to O&M expenses?

Line 3 in Table 10.8.1 above, provides data regarding the affordability of user fees (Almost all of the O&M Committees levying user fees report that all families can afford to pay). No committees were able to provide an estimate of annual routine maintenance costs (likely since this maintenance had been almost wholly labour-based activities) nor were committees able to supply information about their anticipated longer term costs for O&M repair and replacement of system components. It does not appear as though most NCDDP SP O&M Committees are prepared or financially able to undertake longer term O&M and typical capital repair tasks.

Line ministries or government agencies did not contribute to O&M expenses (210 village SP committees report that village contributions constitute 100% of labour and materials).

Recommendation 14: The NCDDP should consider revising O&M Committee documentation to insert specific capital repair estimates. Estimates should be provided appropriate to SP type, for example, roof replacement for buildings, with options described to committees for the funding of such major repair capital works.

10.11 Were necessary Government inputs (e.g., teachers and learning materials for schools, or health workers, drugs and equipment for dispensaries) provided adequately and in a timely manner?

There was a near-uniform feeling expressed by village SP representatives (97% of building SP committees, educators and health workers) that the government was providing them with inputs in an adequate and timely manner for schools and health clinics.

10.12 Are responsibilities, both financial and technical, clearly spelled out for community members and for the government?

Very few O&M Plans (4 SPs) contained clear statements that linked line Ministries to responsibilities for the village infrastructure. The majority (94%) of Plans were found to give a clear idea that villagers would be responsible for the operation and maintenance of the infrastructure themselves.

10.13 Did the community or contractor implementation modalities have any impact on O&M? What investment types are more suitable to community force account in terms of long-term cost effectiveness? What conditions have to be met to make the model of community force account cost effective in the long run?

Table 10.13.1 – CFA SP with user fees in place

Building

(64 SP) Bridge (16) Water Supply (35) Road

(72) Electricity (23) Aver.

1 O&M user fee in place (All SP) 4 2 16 3 13 18%

2 CFA SP with user fee in place 4 2 16 3 12 97%

3 SPs with bank account 8 4 14 11 11 23%

4 CFA SPs with bank account 8 4 14 10 11 98%

Discussion:

The comparison of lines 1 and 2 in Table 10.13.1 shows that almost all SPs that collect user fees were constructed using the CFA modality (97% of SPs with user fees were CFA modality). A single user-fee-collecting electrical SP in Kyunsu used a contractor. This would tend to indicate that those villages choosing to manage the construction of their own SP via CFA modality are more inclined to institute user fees. Still, it must be noted, only 25% of the sampled CFA are currently charging user fees.

Lines 3 and 4 shows how many of those O&M Committees that have funds in a village account were constructed using CFA modality. Again, the analysis shows that all funded O&M Committees are in villages that employed CFA modality, except for a single road SP in Kanpetlet.

It appears evident that villages that use a CFA construction modality are more inclined to institute user fees for community infrastructure and are more likely to hold funds in village accounts for future use in operation and maintenance activities.

Recommendation 15: NCDDP should consider the encouragement of the use of CFA construction modality during its socialization phase in Townships and Village Tracts. NCDDP should seek input from Village SP implementation committees as to how Project support activities can be strengthened in encouragement of the CFA modality.

10.14 Does community capacity development carried out by the NCDDP contribute to SP sustainability cost-effectively? Compare the total cost including the cost of community engagement and capacity development of investments financed by different sources, taking into account (i) the current conditions of infrastructure; (ii) initial condition of infrastructure after completion and (iii) O&M works done. Any indication that the NCDDP’s investments in the capacity development of communities contribute to long-term sustainability of subprojects? If such an indication is observed, how cost effective is the NCDDP community capacity development in long-term sustainability of infrastructure?

Village SP implementation and O&M Committee members were asked about the training and ongoing capacity development that were received as part of the SP construction and hand-over process. These interview results are reported above in Table 10.7.1. Almost all of the SPs evaluated reported receiving O&M training (207 of 210 SPs) while 203 committees said that they receive ongoing assistance in these regards. Since these processes seem to be comfortably in place for almost all NCDDP SPs, it is impossible to make comparisons between those-receiving-training with those-not-receiving in order to draw conclusions regarding cost effectiveness. The three SPs whose village committees did not receive O&M training are in Kyunsu (a road and a bridge) and Tatkon (road).

No specific costs for community engagement or capacity development were available in the SP files examined. An evaluation question regarding O&M Committee training budget returned no information from the 210 SP evaluated – in other words, not a single SP has an identified training budget. The TE teams also attempted to gather information from the CSP visited in regards to community involvement, capacity development, and associated costs. Most of the CSP evaluated had been constructed by DRD, which does not fund community involvement to the same extent as NCDDP, if at all. The initial condition of NCDDP infrastructure after construction was also impossible to gauge at the time of the evaluation. The effect of O&M works done was assessed in the technical audit O&M component (and reported in 10.1)

As noted above, with the evaluation’s finding that almost all NCDDP SPs receive both O&M training during the SP completion process and ongoing capacity development support, no comparisons or conclusions in regards to the contributions of these activities can be formed. It does seem logical, however, that community training and capacity development is a necessary and vital activity that will benefit the long-term sustainability of the rural infrastructures and increase its cost effectiveness.

11 Best Practices and Recommendations

The technical ratings of SP components and aspects, gathered by Field Tool 1, have been discussed at high levels in Section 7 of this report. The technical ratings data were aggregated, sorted and studied on a Township level, according to SP type, by quality of design, by remoteness, by functionality, by construction year, by technical facilitation, and by World Bank rating.

The data can similarly be sorted and studied within each SP type. This section will look at each SP type in turn. A study of the ratings applied to each SP type’s components and aspects will yield valuable insights to NCDDP’s current construction methodologies and how they might be improved in future cycles.

Additional information regarding key design and construction issues was gathered in Field Tool 5, Key Issues. The use of this checklist allowed the technical evaluators to further define problems noted with the various components and aspects of the infrastructure. Where applicable and helpful, the aggregated percentages of these key issues are cited below.

Much of the following analysis and design/construction hints and recommendations have been depicted and described in the recent PowerPoint presentation developed for an NCDDP training session.

11.1 Buildings

Most of the buildings examined during this technical evaluation met the specifications set out for them (79% Meet Spec) or were considered Slightly Below Spec (19%). Only 1% of the building components evaluated were rated Below Spec.

For rating purposes buildings were divided into 21 components/aspects that were individually assessed and rated. An examination of this data shows that those components/aspects most often considered Slightly Below Spec are as shown in the following table. Not all building components and aspects are shown, for brevity.

Table 11.1.1 Building Components/Aspects Considered Slightly Below Spec (Note 1)

Building Component/Aspect

(19 SP) Percentage of SP

Rated Slightly Below Spec

Column 10%

Ring Beam 10%

Truss – Structural 28%

Truss – Connection to Ring Beam 67%

Roof – Sheeting, Tiles 14%

Roof – Connection to Purlin 58%

Ceiling 19%

Doors and windows 26%

Toilet/Septic Tank 25%

Ramp for disabled 44%

Drainage 13%

Note 1: no significant Below Spec items were recorded.

Discussion and Recommendations:

Columns were considered to be Slightly Below Specification if their dimensions do not match the designs or a column spacing wider than as shown on design drawings. A small percentage of SPs (10%) were found lacking in this building component. A Key Issues finding shows that 22% of SPs have constructed dimensions (as-built) smaller than those shown on the approved drawings.

Ring beams are those structural members that connect the columns at the top of building walls. The dimensions and connections of these beams (either wood or reinforced concrete depending on the structural design) is an important facet of the building’s strength in hurricanes or earthquake events. A small percentage (10%) of the ring beams that could be inspected (many were hidden above ceilings) were found deficient is some aspect of their installation.

Trusses were evaluated in regards to two aspects: structural standards and conformance with drawings (28% Slightly Below Spec); and proper connections to a building’s ring beam (67% Slightly Below). Many trusses are constructed by local woodworkers who occasionally fabricate wooden trusses using traditional layout and joinery methods, ignoring design drawings. Connections and structural bracing are sometimes neglected. In other instances, design drawings were lacking sufficient detail, leaving the community to trust the skill of local builders.

The use of proper connections from a building’s trusses to the ring beam is very important. This detail is often vague or missing on NCDDP design drawings. Local builders often use nails to fasten the truss to the ring beam. Nails provide a very weak connection and can be pulled loose during high winds, allowing the roof to ‘lift off’ from the building, causing great damage. The use of bolts to connect the truss to the ring beam or columns of a building is imperative.

Roofs can start to leak within a few years if they have been poorly installed or if other elements of the roof structure allow vibration in the roof sheeting during strong winds (roof sheathing: 14% Slightly Below Spec). Proper fasteners (wind ties, cleats) and attention to correct roof construction methodologies will prolong the life of galvanized sheet steel roofs (roof connection to purlin: 58% Slightly Below).

Aggregated findings from Field Tool 5, Key Issues, also highlight the roof as a problem area: 27% of buildings surveyed had improper roof connections; 39% used nails rather than bolts; 20% improper truss/ring beam connection. These percentages differ slightly from those gathered and analyzed using the technical evaluation field tool (and summarized in Table 11.1.1, for example), which possibly is to be expected from the way in which the technical evaluation teams performed the evaluation. The primary instrument used at the infrastructure sites was Field Tool 1; the other tools were sometimes started there but then completed afterwards or even delayed to the digital input phase of the evaluation.

Ceilings were found Slightly Below Spec in 19% of the buildings evaluated. For the most part this is due to the oversight of including an access portal from the interior of a building to the attic space.

Doors and windows were frequently noted as being Slightly Below Spec (26% of SPs). These ratings are directed at sagging and fractured panels that are only a few years old. Properly constructed doors and window panels, using high-grade wood, should last a decade before needing major repair or refurbishment. The use of lower-grade woods, inadequate millwright techniques and inexpensive hardware serve to cheapen a building for its users.

Sanitation facilities had 25% of their components considered Slightly Below. Notes regarding this topic cited leaking pipes, broken faucets, poorly graded floors that have pools of stagnant water, exposed plastic pipe and poor access to septic tank for inspections and cleaning. Some plans featured inadequate septic tank designs. Septic tank should include two-chamber tanks draining toward a separate open-bottomed soak away pit.

Ramps and accessibility features for the disabled has been discussed in Section 7.10 – Universal Accessibility. The majority of the buildings visited did not feature adequate UA measures.

And finally, drainage around the building SPs was considered to be lacking in 13% of the sites visited. This finding is most often directed at ponded water in the vicinity of walking paths or stagnant pools around the school or public building. The nuisance factor of stepping around puddles and the opportunity for breeding of disease vectors contribute to this low rating. Designers must take note to situate buildings high on sites and provide adequate drainage courses to guide storm runoff away.

It is noted that reinforced concrete is used at very few NCDDP SP building sites. The majority of NCDDP buildings are constructed of wooden columns and beams with mortared brick or cement block wall infill.

The NCDDP building program has produced many fine schools, health clinics and other public structures. Building program engineers and technicians should carefully review the findings of this evaluation, as described in the building components above, and make improvements to future infrastructures in areas noted.

Recommendation 16 – NCDDP engineers who are expert in building design and construction should continue their checks and verification of SP designs of this type. Site inspection visits before, during and after construction should continue.

11.2 Bridges

Foundation, abutment and wingwall design are fundamental to the integrity of a bridge structure and must be based on the actual condition of each individual site. National NCDDP expert engineers review all bridge designs that feature these components. Standard design manuals contain generic drawings and specifications, but these must be carefully chosen and fitted to each individual site. Additional features such as wingwalls, ramp, slope protection, etc. are added during the design stage based on the field survey. Foundation considerations are amongst the most crucial of decisions in bridge planning and design, carefully considering the nature of the underlying soils. Senior personnel should be consulted throughout the design process. Erosion protection measures must be carefully selected, designed, installed, and maintained. Ministry sectors should continue to be consulted and involved with these sub-projects, particularly since use of public equipment might be requested in the future for maintenance and repair activities.

The following table provides an abbreviated list of bridge components, concentrating on those parts of SPs that exhibit problems. Not all components are present in all SPs. For example, though 15 NCDDP bridges were evaluated, only 9 of them require handrails for safety purposes (and only one of those was found to be lacking this component).

Table 11.2.1 Bridge Components Ratings (% and No. of SP)

Bridge Component

Percentage of NCDDP SP

Rated Slightly Below

(No. of SPs) Percentage of NCDDP SP

Rated Below Spec

(No. of SPs)

Layout [14 SPs evaluated] 14% (2) 7% (1)

Erosion Protection [14] 14 % (2) 7% (1)

Concrete [13] 8% (1) 8% (1)

Handrail [9] 11% (1)

Connections: nails, bolts [4] 25% (1)

Discussion and Recommendations:

Fourteen (14) NCDDP bridges were evaluated during the fieldwork. Layout of the infrastructure was found to have some slight problems for 2 of these bridges and larger concerns for a single installation (in village Nyaung Bee, Kyunsu). Layout of bridges is very site-specific. The evaluators decided that the two bridges rated Slightly Below (in Lote Ywa, Ann and Aung Myay Yeik Tha, Tatkon) were over-built for their locations. In both cases the evaluation teams noted cost-saving measures that engineers might have taken during the design of the structures. No photographs were submitted for the Below Spec bridge in Kyunsu and no explanatory notes were provided to explain the rationale for this rating.

Erosion protection measures were inadequately done at three of the 14 sites visited, one of them very poorly done (again in Nyaung Bee, no photos available). The choice and placement of erosion protection materials (free stone, masonry, gabion baskets) differs from site to site. Streambeds need to be observed, preferably during flood periods, so that suitable and adequate protection features can be installed where flowing water and erosion are likely. Each site will be unique and drawings should be developed accordingly.

Most of the reinforced concrete inspected had been properly formed and poured in place. Little honeycombing was visible – only 2 of 13 bridges had porosity issues over small areas of their surfaces. Often, when poor concrete pouring and vibration is observed over limited and localized areas it is indicative of one or two shifts of unskilled labour with inadequate supervision. No inspections of the reinforcing bar fabrication and placement were possible in the completed pieces of infrastructure.

A handrail was found to be missing from a bridge, the same SP in Nyaung Bee, Kyunsu,

And finally, connections were poorly done at one bridge in Lay Thar, Pinlebu. Bolted connections are vital for bridges, as vehicular traffic causes great vibration of structural members. Nails will slowly work loose over time, allowing movement in wooden structures, which leads to splintering and cracking of bridge components. Bolts can be tightened periodically and also allow for easy replacement of bridge components as timbers age.

Recommendation 17: NCDDP engineers who are experts in bridge design should continue their checks and verification of SP designs of this type. Site inspection visits before, during and after construction should continue.

11.3 Water Supply Systems

Similar to bridge SPs above, water supply sub-projects frequently involve specialized knowledge and experience. The relatively high quality of water supply SPs shows that senior NCDDP design/construction engineers have provided expert guidance, assistance and advice to NCDDP field personnel.

Table 11.3.1 Water Supply Component/Aspect Ratings (% and No. of SP)

Water Supply Component/

Aspect Percentage of NCDDP SP

Rated Slightly Below Percentage of NCDDP SP

Rated Below Spec

System Design [30 SPs evaluated] 27% (8) 3% (1)

Watershed protection [29] 28% (8) 7% (2)

Borehole and Pump System [18] 17% (3) 6% (1)

Reservoir – Ease of cleaning [28] 11% (3)

Transmission pipe [18] 56% (10)

Public tap – fixture/platform [12] 33% (4)

Public tap – drainage[11] 36% (4) 18% (2)

Water pressure/quantity [17] 29% (5)

Discussion:

Water system design was judged Slightly Below Spec for 8 of 30 SPs evaluated, and Below Spec at a single village, Zee Kone, Tatkon. Commentary from the field tools for these various SPs provides a variety of reasons for these lower ratings: poor pipe design and installation promotes leakages, engines have not been vented correctly, lack of filtration at source, poor location of water intake, etc.

Watershed protection has not been sufficiently addressed in 8 of 29 SPs evaluated. This can be fencing of upland areas for gravity-fed sources or the provision of proper separation distances within villages for boreholes from unsanitary conditions. A dug well in the middle of paddy fields in Ohn Pin East, Tatkon should spur conversations between the water user group and local farmers in favour of banning the use of pesticides and chemical fertilizers in the immediately adjacent rice fields.

Borehole wells and pumping systems were found to be lacking in almost a quarter of the SPs evaluated. Reasons cited in commentary on the field tools indicate that problems occur is a number of ways: inadequate water-cooling systems, use of plastic pipe rather than galvanized, and poor venting of engines (a common problem).

Plumbing provisions for reservoir cleaning are being included in most NCDDP water supply SPs (89% included these items), but NCDDP engineers should strive to ensure that all reservoirs are equipped with well-designed drains that are easy to access and use. The Key Issue summary of typical problem areas also identified improper overflow pipes for reservoirs (17% of reservoirs evaluated) and poorly constructed valve boxes (19%).

Water transmission pipes (that transport water from the catchment reservoir/tank to the village) have been constructed Slightly Below Spec in 56% of the SPs evaluated (10 of 18). Substandard work in this case normally consists of inadequately supported pipe (improper pipe stands), lack of cover over pipe (especially PVC), or poor assembly of the piping.

Public tapstand platforms and drainage have high percentages of Slightly Below ratings (33% and 36% respectively) while the drainage from 2 tapstands was considered Below Spec. Imperfections are generally associated with faulty faucets, poorly installed and leaking pipes, or improperly graded concrete platforms that allows water to pool (a nuisance for users and potential breeding area of disease vectors).

Water pressure and quantity was identified as problematic at roughly a third of the SPs evaluated. There is sometimes little that can be done about this, due to constraints presented by elevations of sources, spring-fed volumes fluctuating during the year, and limited groundwater recharge. Engineers should be aware of this village concern and should work to ensure that installed systems are as leak-free as possible.

Recommendation 18: NCDDP engineers who are experts in water supply design should continue their checks and verification of SP designs of this type. Site inspection visits before, during and after construction should continue.

11.4 Roads

NCDDP sponsors the construction of roads using several different methodologies and construction materials. These are as follows, with the percentage of each road type as evaluated by this study (per Table 8.1.1.3, Road Construction Materials):

1) Earth road (14%)

2) Gravel road (24%)

3) Macadam surfaced road (6%)

4) Concrete road or concrete wheel paths (57%)

The NCDDP Operations Manual menu provides for rural road rehabilitation. If the proposed road works are in a hilly or mountainous area, proper road design will require a detailed survey, good knowledge of the local soils, awareness of local drainage patterns, and plenty of design experience.

The bulk of NCDDP’s road building works, based on the evaluation sample, has taken place on flat terrain with few major drainage courses crossing the road alignments. These conditions have benefited the construction program and helped to produce some well-constructed village roads.

The road components that received ratings of Slightly Below and Below Spec are as follows:

 

Table 11.4.1 Road Component/Aspect Ratings (% and No. of SP)

Road Component/Aspect

[No. of SPs Evaluated] Percentage of SP rated

Slightly Below Spec Percentage of SP

Rated

Below Spec

Road surface – lack of crown [64] 14% (9)

Road surface – below standard [57] 7% (4)

Inadequate roadside ditches [49] 24% (12) 8% (4)

Unstable slope below [60] 10% (6)

Culvert [7] 43% (3) 14% (1)

Retaining wall – weep holes [3] 67% (2)

Discussion:

Road surface issues, crown and surface standard – the shape and crown of a road cross section is important to properly shed stormwater runoff – 9 of 64 roads were rated Slightly Below Spec, while also the surface of 4 roads was observed to be cracked or broken. Cracked road surfaces are often caused by drainage problems beneath the road. Road sub-base soils must be excavated and shaped to form an adequate camber (providing a crown to the road surface), before placement of road gravels or concrete. Failure to do this will promote water pooling beneath the road, softening the underlying sub-base soils and leading to surface cracking. The NCDDP roads evaluated during this assignment are not yet very old, only one to two years, so that more cracking of the roads’ surfaces is likely to come where proper crowns have not been formed.

Ditches – 24% of NCDDP SP are rated Slightly Below Spec in regards to drainage, with another 8% Below Spec. Properly shaped and adequate roadside drainage is vital to the long-term stability of road surfaces. As described above for road surfaces, care and attention must be directed at ensuring roads are adequately drained. This component, almost more than any other, determines the viability of NCDDP road sub-projects into the future.

Fill slope embankments were found to be Slightly Below Spec in 6 of the 60 road SP evaluated (10%). These ratings identify those slopes that are greater than 1 vertical: 2 horizontal and they normally already show signs of erosion. The construction of retaining walls is one solution to the problem of steep slopes, although two wall SPs (of 3) were noted to lacking drainage weep holes, which may contribute to the walls’ instability in the future.

Culverts also suffer from a lack of proper placement and design. 43% of the rated culverts (7 installations) are rated Slightly Below, with a further 14% considered Below Spec. Poorly located culverts become nightmares for maintenance crews, as they rapidly fill with silt and debris. Conversely, well-designed and properly constructed infrastructure simplifies maintenance activities and strengthens a road.

Recommendation 19: The NCDDP engineers who are experts in road, drainage and retaining wall design should continue their checks and verification of SP designs of this type. Site inspection visits before, during and after construction should continue.

Ministry sectors should continue to be consulted and involved with these sub-projects, particularly since use of public equipment might be requested in the future for maintenance and repair activities.

11.5 Electricity

There were 23 NCDDP electrical SPs evaluated during this assignment. Six of the schemes featured solar panels and distribution networks, four were mini-hydro installations plus distribution, six were generators and distribution, and seven were national grid extension schemes.

Table 11.5.1 Electricity Components/Aspects Ratings (% and No. of SP)

Electricity Component/Aspect

[No. of SPs reporting] Percentage of SP rated

Slightly Below Spec Percentage of SP

Rated

Below Spec

Equipment Installation/Venting [16] 53% (9) 18% (3)

Wiring Connections [20] 70% (14)

Poles – Quality [17] 24% (4)

Poles – Installation [18] 44% (8)

Conductors – Horiz. Sep. (17) 18% (3)

Conductors – Vertical Dist. [19] 11% (2)

Discussion:

Faulty equipment installation or noxious gas venting was found to be problematic in almost three-quarters of the electrical SPs evaluated. Many of these instances were considered Slightly Below Spec, for example: engines unattached to the floor; poorly installed piping to water-cooling chambers; unprotected electrical panels (from weather or from small children). Exhaust gases are frequently (almost always) released inside screened enclosures (producing a Slightly Below rating) but occasionally vented inside a closed building (meriting a Below Spec for safety considerations).

Wiring connections have been improperly performed in a majority of cases – 14 of 20 SPs so rated were considered Slightly Below Spec. Many of these instances are loose and dangling wires with poorly connected junctions. These present an electrical hazard to those working close by, plus a safety concern for children who might play in the vicinity.

Two aspects of electrical poles were considered: quality of the pole and installation practices. These were found to be 24% and 44% Slightly Below Spec, respectively. Photographs of some of the poles that were of concern to the evaluators show short or misshapen poles (some appear to be rough, bent or knotted logs). Installation of poles was also called into question where poles are obviously off-vertical or set very much off centre in a concrete foundation.

Finally, the positioning of the electrical conductors on the pole alignments, both horizontal separations and vertical ground clearances, was considered to be Slightly Below Spec in 18% and 11% of the SPs, respectively. The minimum separation distances for electrical transmission wires depend upon the voltage being transmitted and ensure that short circuits are avoided as wires sway in high winds. Vertical clearances are for the safety of people moving beneath, carrying things that might come into contact with the overhead wiring.

Recommendation 20: Township electrical department engineers should be involved with SPs of this type, including providing design and inspection services. A detailed construction practice guideline should be developed for use by village committees during implementation. Site inspection visits before, during and after construction should continue. The technical notes provided by Saw Evans for each electrical SP evaluated during this audit should be studied closely by NCDDP engineers.

11.6 Design Drawings and Construction Detailing

The Key Issues field tool contained a section that was similar between all SP types, Design. The list of potential key issues that could be selected as problematic at SP sites are as follows, along with the data reported from the NCDDP SPs evaluated.

Table 11.6.1 – Key Issues with Design (% of SPs Evaluated)

Building Bridge Water Supply Road Electrical

Lack of construction details/elevations on dwg 31% 60% 28% 36% 38%

Inaccurate drawings of connection details 17% 33% 31% 14% 48%

Constructed dimensions differ from plan 22% 53% 11% 17%

Discussion:

It is clear from these ratings that NCDDP needs to improve the technical quality of its sub-project design drawings and associated construction specifications.

Recommendation 21: NCDDP should develop a comprehensive set of standard template designs, drawings and details in AutoCAD, and specifications for all SP types; and train its technical personnel in the use and modification of these standard drawings.

12 Summary of Findings and Conclusions

Technical Quality

This Final Report of the 2015 Technical, Cost Effectiveness and Sustainability Audit of the National Community Driven Development Project has found that the sub-project works evaluated in nine Townships to be largely in conformance with the Sub-Project Proposals and the specifications as set out by NCDDP for the infrastructure.

Problems and key construction issues were highlighted by the technical evaluation teams as they rated the various components of each sub-project. Aggregates of this information were assembled, analyzed and presented in this report. The recommendations provided can be used to make improvements to the construction program of NCDDP.

Cost Effectiveness

NCDDP building, bridge, borehole water supply, earth and gravel road, and solar panel electrification sub-projects have been found to be more cost effective than similar infrastructures constructed by other agencies (often DRD). No comparable gravity-fed water schemes were evaluated to help determine the cost effectiveness of NCDDP’s efforts for this type of water supply system. NCDDP’s concrete road building program was found to be slightly less cost effective than similar works by DRD. The study was also unable to find comparable works in macadam road, electrical generator and mini-hydro installations.

Environmental and Social Safeguards

The technical audit of NCDDP SP files in recipient villages showed that in all Townships the vast majority of village implementation committees and their NCDDP handlers are successfully completing ECOP and the Social Safeguards Checklist (about 97% of the village files evaluated contained these documents).

NCDDP policies are being followed in regards to land donation. Over 80% of SPs are constructed on existing public lands. All donated lands were transferred with appropriate documentation in place.

Operation and Maintenance/Sustainability

The current condition of the NCDDP infrastructure is good. Thirty SPs from Year 1 were evaluated with respect to current conditions, state of maintenance, and ongoing sustainability. 87% of SPs were considered Good, with the remaining 13% rated as Fair (with commentary provided for appropriate corrective measures).

The sustainability of the SPs was assessed by measuring the occurrence of routine maintenance by O&M Committee members. For all SP types, the frequency of routine maintenance activities was found to increase as time goes by. That is, Year 1 O&M Committees are more fully participating and performing routine maintenance tasks than Year 2 SP Committees.

User fees are in place in 18% of villages evaluated. A slightly greater number, 23% of villages, have an O&M bank account where maintenance funds are held. The majority of villages collect O&M fees on a sporadic basis for specific repairs that require purchase of materials. Infrastructure user fees, where they exist, are affordable to all villagers.

Conclusions

Some of the results of this technical evaluation are uncertain and possibly inconclusive. Often this is a result of the small sample size or missing information (such as lack of a comparable infrastructure). Special studies that restrict their sampling to certain types of SPs may be valuable to further define problems and opportunities.